Intel® Open Source HD Graphics and Intel Iris™ Plus Graphics

Programmer's Reference Manual

For the 2016 - 2017 Intel Core™ Processors, Celeron™ Processors,

and Pentium™ Processors based on the "Kaby Lake" Platform

Volume 8: Media VDBOX

January 2017, Revision 1.0

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Creative Commons License

You are free to Share - to copy, distribute, display, and perform the work under the following

conditions:

 Attribution. You must attribute the work in the manner specified by the author or licensor (but

not in any way that suggests that they endorse you or your use of the work).

 No Derivative Works. You may not alter, transform, or build upon this work.

Notices and Disclaimers

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. NO

LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS

IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE

FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY

EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING

LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR

INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

A "Mission Critical Application" is any application in which failure of the Intel Product could result,

directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS

FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS

SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES

OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE

ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY,

PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION,

WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE,

OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice. Designers

must not rely on the absence or characteristics of any features or instructions marked "reserved" or

"undefined". Intel reserves these for future definition and shall have no responsibility whatsoever for

conflicts or incompatibilities arising from future changes to them. The information here is subject to

change without notice. Do not finalize a design with this information.

The products described in this document may contain design defects or errors known as errata which

may cause the product to deviate from published specifications. Current characterized errata are

available on request.

Implementations of the I2C bus/protocol may require licenses from various entities, including Philips

Electronics N.V. and North American Philips Corporation.

Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and other countries.

* Other names and brands may be claimed as the property of others.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

iii

Table of Contents

VDBOX Registers ............................................................................................................................. 1

MFX Architecture ............................................................................................................................ 1

MFX Introduction ..................................................................................................................................................... 1

MFC Overview............................................................................................................................................................ 2

Sample Algorithmic Flow ................................................................................................................................. 3

Synchronization Mechanism ........................................................................................................................... 4

Restrictions ............................................................................................................................................................. 5

MFD Overview ........................................................................................................................................................... 6

MFD Memory Interface ................................................................................................................................... 10

MFD Codec-Specific Commands................................................................................................................. 11

MFX State Model .................................................................................................................................................... 11

MFX Interruptability Model ................................................................................................................................ 12

Decoder Input Bitstream Formats ................................................................................................................... 13

AVC Bitstream Formats – DXVA Short ...................................................................................................... 13

AVC Bitstream Formats – DXVA Long ....................................................................................................... 13

VC1 Bitstream Formats – Intel Long .......................................................................................................... 13

MPEG2 Bitstream Formats – DXVA1 .......................................................................................................... 13

JPEG Bitstream Formats – Intel .................................................................................................................... 13

Concurrent Multiple Video Stream Decoding Support ........................................................................... 14

MFX Codec Commands Summary .............................................................................................. 14

MFX Decoder Commands Sequence .............................................................................................................. 18

Examples for AVC .............................................................................................................................................. 18

Examples for VC1 ............................................................................................................................................... 21

Examples for JPEG ............................................................................................................................................. 22

MFX Pipe Common Commands .................................................................................................. 23

Bitplane Buffer ......................................................................................................................................................... 25

Video Codecs ................................................................................................................................. 26

AVC (H.264) .............................................................................................................................................................. 27

AVC Common Commands ............................................................................................................................. 27

AVC Decoder Commands .............................................................................................................................. 27

Session Decoder StreamOut Data Structure ...................................................................................... 28

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

AVC Encoder PAK Commands ...................................................................................................................... 37

PAK Object Inline Data Description ....................................................................................................... 38

Luma Intra Prediction Modes .............................................................................................................. 51

Reference Indices Defined for Each MB Partition Type and Bit Assignment .................... 55

MB Neighbor Availability in Intra-Prediction Modes (IntraPredAvailFlags) ...................... 55

Macroblock Type for Intra Cases ....................................................................................................... 60

Macroblock Type for Inter Cases ....................................................................................................... 64

Macroblock Type Conversion Rules .................................................................................................. 68

Indirect Data Description ........................................................................................................................... 73

Unpacked Motion Vector Data Block ............................................................................................... 74

Packed-Size Motion Vector Data Block ........................................................................................... 78

Macroblock Level Rate Control ............................................................................................................... 80

Theory of Operation Overview............................................................................................................ 82

AVC Encoder MBAFF Support .................................................................................................................. 83

MPEG-2 ...................................................................................................................................................................... 84

MPEG2 Common Commands ....................................................................................................................... 84

MPEG2 Decoder Commands ........................................................................................................................ 85

Indirect Data Description ........................................................................................................................... 85

MPEG2 Encoder PAK Commands ................................................................................................................ 86

PAK Object Inline Data Description – MPEG-2 ................................................................................. 86

MFX HW Interface and DXVA Conversion .......................................................................................... 93

Map DXVA to HW BSpec ....................................................................................................................... 93

Map HW Bspec to DXVA ....................................................................................................................... 95

Video Codec VC-1 .................................................................................................................................................. 97

VC1 Common Commands ............................................................................................................................. 97

VC1 Decoder Commands ............................................................................................................................... 97

Handling Emulation Bytes ......................................................................................................................... 98

VP8 ............................................................................................................................................................................... 99

MFX_VP8_STATISTICS - Encoder Only ....................................................................................................... 99

VP8 Common Commands ........................................................................................................................... 101

VP8 Decoder Commands ............................................................................................................................ 102

VP8 Encoder Commands ............................................................................................................................. 102

JPEG and MJPEG .................................................................................................................................................. 103

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

JPEG Decoder Commands .......................................................................................................................... 103

JPEG Encoder Commands ........................................................................................................................... 106

More Decoder and Encoder ............................................................................................................................ 109

MFD IT Mode Decode Commands .......................................................................................................... 109

Common Indirect IT-COEFF Data Structure .................................................................................... 109

Inline Data Description in MPEG2-IT Mode .................................................................................... 110

Indirect Data Format in MPEG2-IT Mode ......................................................................................... 113

MFX Deblocking Commands ..................................................................................................................... 113

Encoder StreamOut Mode Data Structure Definition .......................................................... 114

PAK Frame Statistics StreamOut ................................................................................................................... 115

PAK Multi-Pass ..................................................................................................................................................... 116

Driver Usage .......................................................................................................................................................... 117

Programming Reference ........................................................................................................... 117

Monochrome Picture Processing .................................................................................................................. 117

Context Switch...................................................................................................................................................... 118

PMSI Support ........................................................................................................................................................ 118

Pipeline Flush ........................................................................................................................................................ 118

MMIO Interface .................................................................................................................................................... 118

Decoder Registers .......................................................................................................................................... 119

Encoder Registers ........................................................................................................................................... 120

MMIO Interface........................................................................................................................................... 121

Row Store Sizes and Allocations ................................................................................................................... 123

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

VDBOX Registers

This section describes the VDBOX Command Memory Interface registers.

MFX Architecture

This section and the following sections of Media VDBOX contain the referential documentation on the

Multi-Format Codecs, or MFX for those series of chips.

MFX Introduction

Multi-Format Codec (MFX) Engine is the hardware fixed function pipeline for decode and encoding. It

includes multi-format decoding (MFD) and multi-format encoding (MFC).

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFC Overview

Multi-Format Codec (MFX) Engine is the hardware fixed function pipeline for decode and encoding. It

includes multi-format decoding (MFD) and multi-format encoding (MFC).

Note: For Gen6, MFC only supports AVC (H.264).

Many decoding function blocks in MFD such as VIP, VMC, IQT, etc, are also used in encoding mode. Two

blocks, FTQ and BSE, are encoding only.

The encoding process is partitioned across host software, the GPE engine, and the MFX engine. The

generation of transport layer, sequence layer, picture layer, and slice header layer must be done in the

host software. GP hardware is responsible for compressing from Slice Data Layer down to all macro-

block and block layers. Specifically, GPE w/ VME acceleration is for motion vector estimation, motion

estimation, and code decision.

MFC is operated concurrently with and independently from the GPE (3D/Media) pipeline with a separate

command streamer. The two parallel engines have similar command protocol. They can be executed in

parallel with different context. For encoding, motion search, MB mode decision, and rate control are

performed using GPE pipeline resources.

MFC is implemented to achieve the following objectives:

 Compliant with next generation high definition optical video disc requirements, with sufficient

performance headroom:

 Support AVC 4:2:0 Main Profile and High Profile only (8-bit only), up to Level 4.1 resolution

and up to 40 mbps bitstream. With sufficient duty cycles, higher bit rate contents can also be

encoded. There is no support for Baseline, Extended, or High-10 Profiles.

 Performance requirements with MFX core frequency above 667MHz:

 Real-time performance with 20% duty cycle or less.

 Support concurrent decoding of two active HD bitstreams of different formats (for example,

one AVC and one VC1 HD bitstream) and one active HD encoding.

As the result of this hardware partitioning, VPP and ENC are always running in GPE, and PAK is what runs

exactly in MFC.

PAK – residue packing and entropy coding, including block transformation, quantization, data prediction,

bitrate tuning and reference decoding. It delivers final packed bitstream and decoded key-frame

reference:

 As the same as ENC, PAK is invoked on a Slice boundary; a single call of VPP can lead to multiple

calls for PAK.

 Rate control is inside ENC and PAK only, not in VPP.

 PAK must always perform with reconstructed reference picture.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

There is a general dependency of the three operation pipelines. Semaphores are inserted either

according to frames or slices. The main CS will also be notified when the decoded reference is ready for

the next frame set to be encoded. The detailed discussion will be found in a later section.

Host software is responsible for encoding the transport stream and all the sequence, picture, and slice

layer/header in the bit-stream; the MFC system is responsible for compressing from Slice Data Layer

down to all macro-block and block layers.

Sample Algorithmic Flow

Assuming all the hardware components are given, there are infinite usage possibilities left with intention

for software to decide according to its own application needs depending upon the balanced requirement

of coding speed, frame latency, power-consumption, and video quality, and depending upon the usage

modes and user preferences (such as low-frame-rate-high-frame-quality vs. high-frame-rate-low-frame-

quality).

The last part of this chapter, we illustrate a generic sample to show how a compression algorithm can be

implemented to use our hardware.

Step 1. Application or driver initializes the encoder with desired configuration, including speed, quality,

targeted bit-rate, input video info, and output format and restrictions.

Step 2. VPP – Application or driver feeds VPP one frame at a time in coded order with specified frame or

field type, as well as transcoding information: motion vectors, coded complexity (i.e. bit size).

It will perform denoising and deblocking based on original and targeted bit-rate, and output additional

4 spatial variances and 2 temporal variances for each macroblock as well as the whole frame.

Step 3. ENC – Application or driver feeds ENC one coding slice buffer at a time including all VPP output.

The frame level data is accessible to all slices.

a. Encoding setup unit (ESE) will set picture level quality parameters (including LUTs, and other

costing functions) and set target bit-budget (TBB) and maximal bit-budget (MBB) to each

macroblock based on rate-control (RC) scheme implemented. For B-frames, it will also make ME

searching mode decision (either Fast, Slow or Uni-directional).

b. Loop over all macroblocks: calculate searching center (MVP) perform individual ME and IE (MEE).

Multi-thread may be designed for HW according to a zigzag order for minimal dependency issue.

c. ENC make microblock level code decision (CD) outputs macroblock type, intra-mode, motion-

vectors, distortions, as well as TBBs and MBBs.

Step 4. PAK – Application or driver feeds PAK one array of coded macroblocks covering a slice at a time,

including all ENC output. Original frame buffer and reconstructed reference frame buffers are also

available for PAK to access.

a. PAK may create bitstreams for all sequence, gop, picture, and slice level headers prior the first

macroblock.

b. Loop over all macroblocks, accurate prediction block is constructed for either inter- or intra-

predictions (VMC & VIP). If MB distortion is less than some predetermined threshold, for a B slice

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

this step can be skipped as well as the Steps (c)-(e) and jump directly to Step (f); for a key slice the

prediction calculated here will be directly used as the reference thus it jumps to Step (e) after this

step.

c. Differencing the predicted block from the original block derives the residue block. Forward

transformation and quantization (FTQ) is performed. For B slice, it will jump to Step (f) right after.

For other types of slice, Steps (d) and (e) can be performed in a thread in parallel with Step (f) and

beyond.

d. This is for accurate construction of reference pictures. Inverse quantization and inverse

transformation (IQT) are performed and added to the predictions to have the decoded blocks.

e. ILDB is applied accordingly to the reconstructed blocks.

f. Meanwhile macroblock codes: including its configuration info (types and modes), motion info

(motion vectors and reference ids), and residual info (quantized coefficients), are collected for

packing (BSE) in the following sub-steps:

i. Code clean-up (in MPR). Check and verify Mbtype and Cbps, use Skip or Zero respectively if

one can. In principal, when there are equivalent codes, use the simple one.

ii. Drop dependency (in MPR). Calculate relative codes from the absolute codes by associate

them with neighborhood information. All neighborhood correlations are solved in this step.

iii. Unify symbols (in SEC). Translate relative codes into symbols, and table or context indices

that are independent of the concept of syntax type.

iv. Entropy coding (VLE) on symbols.

g. Parsing bitstream data in RBSP form (in VLE), and output to application or driver.

h. By the end of each picture, write out the accurate actual data size to designate buffer for ENC to

access.

Synchronization Mechanism

Encoding of a video stream can be broken down to three major steps (as explained in the previous

section):

1. VPP: video-stream pre-processing

2. ENC: encoding, that is, code decision of inter-MVs and intra-modes

3. PAK: bit-stream packing

a. residual calculation, transformation, and quantization

b. code bit-stream packing

c. reference generation of keyframes

This section describes an architectural solution to map the first two steps in the GFX engine and the last

step in the MFX engine. Since this mapping involves two OS-visible engines, managing them in parallel

under one application is similar to the solution in earlier generations. Each engine has its own command

streamers and has mechanisms to synchronize at required levels as described in the next sub-section.

Above three steps of encoding have dependencies in processing based on

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

i. Functional pipeline order, i.e. on a given frame, VPP needs to be performed first, then ENC,

then PAK and finally MFD (Multi-Format Decoding) for key reference frame generation.

ii. I-frames are key frames for P and B, they have to be first in every pipe-stage.

iii. P-frames are key frames for B frames and therefore P frames are processed first before the

dependent B frames

iv. GFX Engine is time slice to work on either VPP or ENC frame as we discussed in the previous

chapter.

v. PAK + MFD are executed on the same frame in the MFX engine by macro-block level

pipelining within a slice. It should be noted that for the sake of simplicity, an entire frame

(potentially multiple slices) are processed in the corresponding engine and no smaller

granularity of switching is allowed between the functional pipeline stages.

Three steps of the encoding can be interleaved on two engines in the following way on a frame by frame

basis.

Command Stream Synchronization

Restrictions

MFC implementation is subject to the following limitations.

 Context switching within MFC and with Graphics Engine occurs only at frame boundary to

minimize the amount of information that needs to be tracked and maintained.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFD Overview

When used for decoding, we also refer to the MFX Engine as the MFD Engine.

The Multi-Format Decoder (MFD) is a hardware fixed function pipeline for decoding the three video

codec format and one image compression codec format: AVC (H.264), VC-1, MPEG2, and JPEG.

 Compliant with next generation high definition optical video disc requirements, with sufficient

performance headroom:

o Support AVC 4:2:0 Main and High (8-bit only) Profile only (no support for Baseline, Extended

and High-10 Profiles), up to Level 5.1 (max 983,040 MB/s, max 36,864 MB/frame, and at most

one dimension can reach 4K pixel) resolution and up to 40 mbps bitstream. With sufficient

duty cycles, higher bit rate contents can also be decoded.

 Allow a B-picture (frame or field) as a reference picture

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

o Support MVC 4:2:0 Stereoscopic Progressive Profile only, up to Level 5.1 (max 983,040 MB/s

per view, max 36,864 MB/frame per view, and at most one dimension can reach 4K pixel)

resolution and up to 40 mbps bitstream. With sufficient duty cycles, higher bit rate contents

can also be decoded.

 Does not support interlaced video specified in the Stereoscopic Profile

o Support VC1 4:2:0 Simple, Main and Advanced Profiles, up to Level 4 (max 491,520 MB/s and

max 16,384 MB/frame; max only one dimension will be at 4K pixel) resolution and up to 40

mbps bitstream. With sufficient duty cycles, higher bit rate contents can also be decoded.

 Allow a B-field as a reference picture only in interlaced field decoding, no other

modes.

o Support MPEG2 HD Main Profile (4:2:0), up to High Level (1920x1152 pixels) and up to 80

mbps bitstream. With sufficient duty cycles, higher bit rate contents can also be decoded. No

support for SNR and spatial-scalability.

 Does not support B-picture as a reference picture.

o Support Baseline JPEG with five choma types (4:0:0, 4:1:1, 4:2:2, 4:2:0, and 4:4:4. No support

for Extended DCT-based mode, Progressive mode, Loseless mode, nor Hierarchical mode.

 H/W support 64Kx64K, but Surface State can support only up to 16kx16k

Features

Supported

Unsupported

Coding processes

Baseline sequential mode:

 8-bit pixel

precision of source

images

 loadable 2 AC and

2 DC Huffman

tables

 3 loadable

quantization

matrix for Y, U, V

 Interleaved and

non-interleaved

Scans

 Single and

multiple Scans

Extended DCT-based mode, Lossless, Hierarchical modes:

More than 8 bit pixel resolution, progressive mode, arithmetic

coding, 4 AC and 4 DC Huffman tables (extended mode),

predictive process (lossless), multiple frames (hierarchical)

Number of image

channels

1 for grey image

3 for Y, Cb, Cr color

image

4-th channel (usually alpha blending image)

Image resolution

Arbitrary image size up to

16K * 16K

Larger than 16K * 16K (64K * 64K is max. in the JPEG standard)

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Features

Supported

Unsupported

Chroma

subsampling ratio

Chroma 4:0:0 (grey

image)

Chroma 4:1:1

Chroma 4:2:0

Chroma horizontal 4:2:2

Chroma vertical 4:2:2

Chroma 4:4:4

Any other arbitrary ratio, e.g., 3:1 subsampled chroma

Additional feature

(post-processing)

Image rotation:

90/180/270 degrees

o H/W does not impose restriction on picture frame aspect ratio, but is bounded by a max 256

MBs (4096 pixels) per dimension programmable at the H/W interface specifications.

 For example, supporting HD video resolution 1920x1080/60i, 1920x1080/24p,

1280x720/60p

 Performance requirements with MFX core frequency above 1GHz

o Real-time performance around 10% duty cycle

o Support concurrently decoding of at least two active HD bitstreams of different formats (For

example, one AVC and one VC1 HD bitstream)

 The parsing of transport layer and sequence layer is not performed in this hardware, and is

required to be done in the host software.

 The MFD hardware pipeline is operated concurrently with and independently from the Graphics

(3D/Media) pipeline with separate command streamer. The two parallel engines are designed with

the similar command protocol. They can be executed in parallel with different context.

 Local storages and buffers along the hardware pipeline are kept at minimum. For example, there is

no on-die row-store memory. They are resided on the system memory. MFD is designed to hide

the memory access latency (in both the row stores and in the motion compensation units) in

maximizing its decoding throughput.

 Support the following operating modes:

o VLD mode - operation starts from entropy decoding of the compressed bit stream (parsing

Slice Header and Slice Data Layer in AVC, Picture layer, Slice layer and MB Layer in VC-1, and

MB-layer in MPEG2), all the way, to the reconstruction of display picture, including in-loop

and out-loop deblocking, if any.

 Streamout mode - a new feature of the VLD mode in assisting transcoding during

decoding. Selected uncompressed data (e.g. per MB MV information) will be sent out

to the EU and the ME engine (resided on the Sampler of the 3D Gx Pipeline) for

encoding into a different format or for the purpose of transcaling and transrating. In

addition, the uncompressed result may continue to be processed by the rest of

pipeline as in VLD mode to generate the display picture for transcoding. That is, while

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

intermediate data are streaming out to the memory, the MFD Engine continues its

decoding as ususal.

 For JPEG, only VLD mode is supported (No IT mode). Host software decodes Frame

and Scan layers (down to Scan header in the JPEG bit stream syntax) and sends all the

corresponding information and Scan payload to the MFD hardware pipeline.

o IT mode - when host software has already performed all the bit stream parsing of the

compressed data and packaging the uncompressed result into a specific format (as a

sequence of per-MB record) stored in memory. The hardware pipeline will fetch one MB

record at a time and perform the rest of the decoding process as in VLD mode

o Host software (Application) is responsible for parsing and decoding all the transport and

program layers, and all sequence layers. These parameters are passed to Driver and

forwarded to H/W as needed through different STATE commands. Host software is also

responsible for separating non-video data (audio, meta and user data) from sending to H/W.

 MFD Engine is only responsible for macro-block and block layers decoding, plus

certain level of header decoding. For AVC MFD starts decoding from Slice Header; for

VC1, MFD starts decoding from Picture Header, and for MPEG2 decoding starts from

MB Layer only.

 For JPEG, MFD is responsible for ECS and block layers decoding.

 Support bitstream formats (compressed video data) for each codec

o AVC - 2 formats

o MVC - 2 formats

 DXVA2 MVC Short Slice Format

 DXVA2 AVC Long Slice Format Specification (exactly the same as AVC)

o VC1 - 2 formats

 Fully compliant to Picture Parameter and Slice Control Parameter interface definition

o MPEG2

 MB Layer only, according to DXVA 1 Specification

o JPEG

 ECS Layer

 The MFX codec is designed to be a stateless engine, that it does not retain any history of settings

(states) for the encoding/decoding process of a picture. Hence, driver must issue the full set of

MFX picture state command sequence prior to process each new picture. In addition, driver must

issue the full set of Slice state command sequence prior to process a slice.

o In particularly, RC6 always happens between frame boundaries. So at the beginning of every

frame, all state information needs to be programmed. There is no state information as part

of media context definition.

 To activate the AVC deblocker logic for incoming uncompressed 4:2:0-only video stream, one can

pack the uncompressed video stream to compliant with the IPCM MB data format (including ILDB

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

control information) and feed them into the MFD engine in IT mode. Since the MFD Engine is in

IPCM mode, transformation, inter and intra processing are all inactive.

Start Code Detection and removal are done in the CPU, but the Start Code Emulation Prevention Byte is

detected and removed by the front end logic in the MFD. The bitstream format for each codec and for

each mode is specified in this document.

Codec specific information are based on the following released documents from third parties:

 Draft of Version 4 of H.264/AVC (ITU-T Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4

part 10) Advanced Video Coding); JVT-O205d1.doc; dated 2005-05-30

 Final Draft SMPTE Standard : VC1 Compressed Video Bitstream Format and Decoding Process,

SMPTE 421M, dated 2006-1-6; PDF file.

 MPEG2 Recommendation ITU T H.262 (1995 E), ISO/IEC 13818-2: 1995 (E); doc file.

 Digital Compression and Coding of Continuous-tone Still Images, ITU-T Rec. T.81 and ISO/IEC

10918-1: Requirements and guidelines September 18 1992; itu-t81[1].pdf

MFD Memory Interface

The Memory Arbitrator follows the pre-defined arbitration policy (as indicated in the following listing P0

to P11, in which P0 is the highest priority) to select the next memory request to service, then it will

perform the TLB translation (translation to physical address in memory), and make the actual request to

memory.

The Memory Arbitration unit is also responsible for capturing the return data from memory (read

request) and forward it to the appropriate unit along the MFD Engine.

 Read streams: (all 64B requests)

 Commands for BSD : linear ( including indirect data) (P0)

 Indirect DMA (P1)

 Row store for BSD: linear (P5)

 Row store for MPR: linear (P6)

 MC ref cache fetch : tiled (P2)

 Intra row store: linear (P9)

 ILDB row store: linear (P10)

 Write streams: (all 64B requests)

 Row store write for BSD: linear and can avoid partial writes (P3)

 Row store write for MPR: linear and can avoid partial writes (P4)

 Intra row store write: linear and can avoid partial writes (P7)

 ILDB row store write: linear and can avoid partial writes (P8)

 Final dest writes: tiled and can potentially be partial, two ways to avoid these partials: 1)

either write garbage and buffers are aligned or 2) read-modify writes for dribble end of line

cases (P11)

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFD Codec-Specific Commands

MFD hardware pipeline supports 3 different codec standards: AVC, VC1 and MPEG2. To make the

interface flexible, each codec is designed with its own set of commands.

There are two categories of commands for each codec format: one set for VLD mode and one set for IT

mode.

MFX State Model

The parallel video engine (PVE) supports two state delivery models: inline state model and indirect state

model. For inline state model, the state commands (*_STATE) can be issued in batch buffers or ring

buffers directly preceding object commands (*_OBJECT). In the indirect state model, the state commands

are not placed in the batch buffers or ring buffers. Instead Indirect State Buffers provide state

information (in the form of the above mentioned state commands) for the MFX pipeline. See

MFX_STATE_POINTER for more details.

VCS (aka BCS) handles the difference of the two state delivery models. Therefore, the MFX pipeline

always sees the state commands in both models. However, MFX hardware supports additional context

save/restore of ‘dynamic states’. Dynamic states are the internal signals that are persistent. This could be

the CABAC context for macroblock encoding.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFX State Model

The MFX codec is designed to be a stateless engine, that it does not retain any history of settings (states)

for the encoding/decoding process of a picture. Hence, driver must issue the full set of MFX picture state

command sequence prior to process each new picture. In addition, driver must issue the full set of Slice

state command sequence prior to process a slice.

 In particular, RC6 always happens between frame boundaries. So at the beginning of every frame,

all state information needs to be programmed. There is no state information as part of media

context definition.

MFX Interruptability Model

MFX encoding and the encoding pipeline do not support interruption. All operations are frame based.

Interrupts can only occur between frames; the driver will submit all the states at the beginning of each

frame. Any state kept across frames is in MMIO registers that should be read between frames.

Software submits without any knowledge of where the parser head pointer is located. Also there is a

non-deterministic amount of time for the new context to reach the command streamer. However, the

state model for the MFX engine requires software to know exactly what state the pipeline is in at all

times. This introduces cases where a preemption could occur during or after a state change without

software ever knowing the state saved out to memory on the context switch.

Also, preemption is only allowed during the last macroblock in a row. Hardware cannot always perform a

context switch when the new context is seen by the hardware. To avoid a switch during an invalid

macroblock and to keep the state synchronized with software, there are two commands available that are

used. MI_ARB_ON_OFF disables and enables preemption while MFX_WAIT ensures the context switch, if

needed, preempts during macroblock execution. Below illustrates an example assuming VC1 VLD mode.

Command Ring/Batch

Notes

MI_ARB_ON_OFF = OFF

Disable preemption

Inline or indirect state cmd 1

Inline or indirect state cmd 2

Inline or indirect state cmd 3

XXXX_OBJECT

Slice

MI_ARB_ON_OFF = ON

Enable preemption

MFX_WAIT

Allow preemption to occur while XXXX_OBJECT executes

MI_ARB_ON_OFF = OFF

Since arbitration is off again, state commands are allowed below

Inline or indirect state cmd 4

Inline or indirect state cmd 5

Inline or indirect state cmd 6

XXXX_OBJECT

Slice

MI_ARB_ON_OFF = ON

Enable preemption

MFX_WAIT

Allow preemption to occur while XXXX_OBJECT executes

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Command Ring/Batch

Notes

MI_ARB_ON_OFF = OFF

Since arbitration is off again, state commands are allowed below

Note that store DW commands may execute inside the preemption enabling window if needed.

Decoder Input Bitstream Formats

AVC Bitstream Formats – DXVA Short

Bitstream Buffer Address starts after the 3-byte start code, i.e. starts (and includes) at the NAL Header

Byte. This byte must not be included in the Emulation Byte Detection Process.

AVC Bitstream Formats – DXVA Long

Bitstream Buffer Address starts after the 3-byte start code, i.e. starts (and includes) at the NAL Header

Byte. This byte must not be included in the Emulation Byte Detection Process. Application will provide

the Slice Header Skip Byte count (not including any possible Emulation Prevention Byte).

VC1 Bitstream Formats – Intel Long

Bitstream starts right at the MB layer, with any emulation byte crossing the header and MB layer being

removed by application and the data length is adjusted.

MPEG2 Bitstream Formats – DXVA1

Bitstream buffer starts right at the very first MB data.

JPEG Bitstream Formats – Intel

Bitstream buffer starts right at the very first MCU data of each Scan.

The indirect data start address in MFD_JPEG_BSD_OBJECT specifies the starting Graphics Memory address

of the bitstream data that follows the Scan header. It provides the byte address for the first MCU of the

Scan. Different from MFD_MPEG2_BSD_OBJECT command, First MCU Bit Offset does not need to be

specified because it is always set to zero.

Indirect data buffer for a Scan

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

The indirect data length in MFD_JPEG_BSD_OBJECT provides the length in bytes of the bitstream data for

the Scan excluding Scan header. It includes the first byte of the first macroblock and the last byte of the

last macroblock in the Scan. The Figure illustrates these parameters for a slice data.

Concurrent Multiple Video Stream Decoding Support

The natural place for switching across multiple streams is at the Slice boundary. Each Slice is a self-

sustained unit of compressed video data and has no dependency with its neighbors (except for the

Deblocking process). In addition, there is no interruptability within a Slice. However, when ILDB is

invoked, the processing of some MBs will require neighbor MB information that crosses the Slice

boundary. Hence, to limit the buffering requirement, in this version of hardware design, stream switching

can only be performed at the picture boundary instead.

MFX Codec Commands Summary

DWord

Bit

Description

31:29

Instruction Type = GFXPIPE = 3h

28:16

3D Instruction Opcode = PIPELINE_SELECT

GFXPIPE[28:27 = 1h, 26:24 = 1h, 23:16 = 04h] (Single DW, Non-pipelined)

15:1

Reserved: MBZ

Pipeline Select

0: 3D pipeline is selected

1: Media pipeline is selected

Pipeline Type (28:27)

Opcode (26:24)

Sub Opcode (23:16)

Command

Definition Chapter

VC1 State

VC1_BSD_PIC_STATE

VC1 BSD

Reserved

n/a

Reserved

n/a

VC1_BSD_BUF_BASE_STATE

VC1 BSD

Reserved

n/a

5h-7h

Reserved

n/a

VC1 Object

VC1_BSD_OBJECT

VC1 BSD

9h-FFh

Reserved

n/a

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Pipeline Type (28:27)

Opcode (26:24)

Sub Opcode (23:16)

Command

Definition Chapter

State

2h-7h

Reserved

N/A

Object

9h-FFh

Reserved

N/A

Note that it is possible for a command to appear in both IMAGE and SLICE state buffer, e.g. QM_STATE

for JPEG can be issued at frame level or scan/slice level.

Pipeline

Type

(28:27)

Opcode

(26:24)

SubopA

(23:21)

SubopB

(20:16)

Command

Chapter

Recommended

Indirect State

Pointer Map

Interruptable?

MFX

Common

MFX_PIPE_MODE_SELECT

MFX

IMAGE

MFX_SURFACE_STATE

MFX

IMAGE

MFX_PIPE_BUF_ADDR_STATE

MFX

IMAGE

MFX_IND_OBJ_BASE_ADDR_STATE

MFX

IMAGE

MFX_BSP_BUF_BASE_ADDR_STATE

MFX

IMAGE

MFX_ STATE_POINTER

MFX

IMAGE

MFX_QM_STATE

MFX

IMAGE/SLICE

MFX_FQM_STATE

MFX

IMAGE

MFX_DBK_OBJECT

MFX

IMAGE

A-1Eh

Reserved

n/a

MFX

Common

Dec

0-8h

Reserved

n/a

MFD_ IT_OBJECT

MFX

n/a

A-1Fh

Reserved

n/a

MFX

Common

Enc

0-7Fh

Reserved

n/a

MFX_PAK_INSERT_OBJECT

MFX

n/a

Reserved

n/a

MFX_STITCH_OBJECT

MFX

n/a

B-1Fh

Reserved

n/a

AVC/

MVC

Common

(State)

MFX_AVC_IMG_STATE

MFX

IMAGE

n/a

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Pipeline

Type

(28:27)

Opcode

(26:24)

SubopA

(23:21)

SubopB

(20:16)

Command

Chapter

Recommended

Indirect State

Pointer Map

Interruptable?

Reserved

n/a

MFX_AVC_DIRECTMODE_STATE

MFX

SLICE

n/a

MFX_AVC_SLICE_STATE

MFX

SLICE

n/a

MFX_AVC_REF_IDX_STATE

MFX

SLICE

n/a

MFX_AVC_WEIGHTOFFSET_STATE

MFX

SLICE

n/a

Reserved

n/a

D-1Fh

Reserved

n/a

AVC/

MVC

Dec

0-5h

Reserved

MFX

n/a

MFD_AVC_DPB_STATE

MFX

IMAGE

n/a

MFD_AVC_SLICEADDR_OBJECT

MFX

n/a

MFD_AVC_BSD_OBJECT

MFX

n/a

9-1Fh

Reserved

n/a

AVC/

MVC

Enc

0-8h

Reserved

n/a

MFC_AVC_PAK_OBJECT

MFX

n/a

A-1Fh

Reserved

n/a

AVC/

MVC

Extension

VC1

Common

(State)

Reserved

n/a

MFX_VC1_PRED_PIPE_STATE

MFX

IMAGE

n/a

MFX_VC1_DIRECTMODE_STATE

MFX

SLICE

n/a

3-1Fh

Reserved

n/a

VC1

Dec

MFD_VC1_SHORT_PIC_STATE

MFX

IMAGE

n/a

MFD_VC1_LONG_PIC_STATE

MFX

IMAGE

n/a

2-7h

Reserved

n/a

MFD_VC1_BSD_OBJECT

MFX

n/a

9-1Fh

Reserved

n/a

VC1

Enc

0-1Fh

Reserved

n/a

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Pipeline

Type

(28:27)

Opcode

(26:24)

SubopA

(23:21)

SubopB

(20:16)

Command

Chapter

Recommended

Indirect State

Pointer Map

Interruptable?

MPEG2

Common

(State)

MFX_MPEG2_PIC_STATE

MFX

IMAGE

n/a

1-1Fh

Reserved

n/a

MPEG2

Dec

1-7h

Reserved

n/a

MFD_MPEG2_BSD_OBJECT

MFX

n/a

9-1Fh

Reserved

n/a

MPEG2

Enc

0-2h

Reserved

n/a

MFC_MPEG2_PAK_OBJECT

3-8h

Reserved

MFC_MPEG2_SLICEGROUP_STATE

A-1Fh

Reserved

VP8

Common

(State)

MFX_VP8_PIC_STATE

MFX

IMAGE

n/a

VP8

Dec

MFD_VP8_BSD_OBJECT

MFX

IMAGE

VP8

Enc

Reserved

JPEG

Common

MFX_JPEG_PIC_STATE

MFX

IMAGE

Reserved

n/a

MFX_JPEG_HUFF_TABLE_STATE

MFX

IMAGE

3-1Fh

Reserved

n/a

JPEG

Common

MFX_JPEG_PIC_STATE

MFX

IMAGE

Reserved

n/a

MFX_JPEG_HUFF_TABLE_STATE

MFX

IMAGE

3-1Fh

Reserved

n/a

JPEG

Dec

1-7h

Reserved

MFX

n/a

MFD_JPEG_BSD_OBJECT

MFX

MCU

9-1Fh

Reserved

MFX

n/a

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Pipeline

Type

(28:27)

Opcode

(26:24)

SubopA

(23:21)

SubopB

(20:16)

Command

Chapter

Recommended

Indirect State

Pointer Map

Interruptable?

JPEG

Enc

0-1Fh

Reserved

MFX

n/a

MMIO Space Registers

Range Start

Range End

Unit owner

00002000

00002FFF

Render/Generic Media Engine

00004000

00004FFF

Render/Generic Media Graphics Memory Arbiter

00005000

0000517F

00006000

00007FFF

Reserved

00012000

000123FF

MFX Control Engine (Video Command Streamer)

00012400

00012FFF

Media Units (VIN unit)

00014000

00014FFF

MFX Memory Arbiter

00022000

00022FFF

Blitter Engine

00024000

00024FFF

Blitter Memory Arbiter

00030000

0003FFFF

Reserved

00100000

00107FFF

Fence Registers

00140000

0017FFFF

MCHBAR (SA)

Memory Interface Command Map

04h Opcode (28:23)

MI_FLUSH

MFX Decoder Commands Sequence

The MFX codec is designed to be a stateless engine, that it does not retain any history of settings (states)

for the encoding/decoding process of a picture. Hence, driver must issue the full set of MFX picture state

command sequence prior to process each new picture. In addition, driver must issue the full set of Slice

state command sequence prior to process a slice.

In particular, RC6 always happens between frame boundaries. So at the beginning of every frame, all

state information needs to be programmed. There is no state information as part of media context

definition

Examples for AVC

The following gives a sample command sequence programmed by a driver

a) For Intel or DXVA2 AVC Long Slice Bitstream Format

MFX_PIPE_MODE_SELECT

MFX_SURFACE_STATE

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFX_PIPE_BUF_ADDR_STATE

MFX_IND_OBJ_BASE_ADDR_STATE

MFX_BSP_BUF_BASE_ADDR_STATE

MFX_QM_STATE

VLD mode: MFX_AVC_PICID_STATE

MFX_AVC_IMG_STATE

MFX_AVC_DIRECTMODE_STATE

MFX_AVC_REF_IDX_STATE

MFX_AVC_WEIGHTOFFSET_STATE

MFX_AVC_SLICE_STATE

VLD mode: MFD_AVC_BSD_OBJECT

IT mode: MFD_IT_OBJECT

MI_FLUSH

b) For DXVA2 AVC Short Slice Bitstream Format (for VLD mode only)

MFX_PIPE_MODE_SELECT

MFX_SURFACE_STATE

MFX_PIPE_BUF_ADDR_STATE

MFX_IND_OBJ_BASE_ADDR_STATE

MFX_BSP_BUF_BASE_ADDR_STATE

MFD_AVC_DPB_STATE

VLD mode: MFX_AVC_PICID_STATE

MFX_AVC_IMG_STATE

MFX_QM_STATE

MFX_AVC_DIRECTMODE_STATE

VLD mode : MFD_AVC_SLICEADDR_OBJECT

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

VLD mode: MFD_AVC_BSD_OBJECT

VLD mode : MFD_AVC_BSD_SLICEADDR_OBJECT

VLD mode: MFD_AVC_BSD_OBJECT

… repeat these four commands N-1 times for a N-slice picture

VLD mode: MFD_AVC_BSD_OBJECT (for the last slice of the picture)

MI_FLUSH

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Examples for VC1

The following gives a sample command sequence programmed by a driver

a) For Intel Proprietary Long Bitstream Format

MFX_VC1_DIRECTMODE_STATE

MFX_VC1_PRED_PIPE_STATE

MFX_VC1_LONG_PIC_STATE

VLD mode: MFD_VC1_BSD_OBJECT

IT mode: MFD_IT_OBJECT

MI_FLUSH

b) For DXVA2 VC1 Compliant Bitstream Format (for VLD mode only)

MFX_VC1_DIRECTMODE_STATE

MFX_VC1_PRED_PIPE_STATE

MFX_VC1_SHORT_PIC_STATE

VLD mode: MFD_VC1_BSD_OBJECT

MI_FLUSH

c) For DXVA2 VC1 Compliant Bitstream Format (for VLD mode only), and field pair picture

Batch buffer for top-field

states....

Slice_objs...

MI_flush

store register immediate (if VC1 short format with interlaced field pic)

MI_flush

Batch buffer for bottom field

load register immediate (if VC1 short format with interlaced field pic)

MI_flush

states....

Slice_objs...

MI_flush

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Examples for JPEG

The following gives a sample command sequence programmed by a driver

Programmed once at the start of decoding

MFX_PIPE_MODE_SELECT

MFX_PIPE_SURFACE_STATE

MFX_IND_OBJ_BASE_ADDR_STATE

MFX_PIPE_BUF_ADDR_STATE

MFX_JPEG_PIC_STATE

Programmed at the start of Frame or Scan (These commands can be sent multiple times either before

MFX_JPEG_PIC_STATE or before MFD_JPEG_BSD_OBJECT)

MFX_JPEG_HUFF_TABLE

MFX_QM_STATE

Programmed per Scan (These commands can be sent multiple times depending on each bit stream)

MFD_JPEG_ BSD_OBJECT

MI_FLUSH

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFX Pipe Common Commands

MFX Commands are organized into groups based on their scope of functioning. There are Pipeline

Common state commands that are common to all codecs (encoder and decoder) and is applicable to the

processing of one full frame/field. There are also individual codec Common state commands that are

common to both encoder and decoder of that particular codec. These latter common state commands,

some are applicable at the processing of one full frame/field, and some are applicable at the processing

of an individual slice level.

MFX_STATE_POINTER

MFX_PIPE_MODE_SELECT

The Encoder Pipeline Modes of Operation (Per Frame):

1. PAK Mode: VCS-command driven, setup by driver. Like the IT mode of decoder, it is executed on a per-MB

basis. Hence, each PAK Object command corresponds to coding of only one MB.

a. Normal Mode (including transcoding): receive per-MB control and data (MV, mb_type, cbp, etc.).

It generates the output compressed bitstream as well as the reconstructed reference pictures, one

MB at a time, for later use.

b. Encoder StreamOut Mode: to provide per-MB, per-Slice and per-Frame coding result and

information (statistics) to the Host, Video Preprocessing Unit and ENC Unit to enhance their

operations.

The Decoder Pipeline Modes of Operation (Per Frame):

1. VLD Mode: The output from the BSD (weight&offset/coeff/motion vectors record) can be sent in part (as

specified) and to the remaining fixed function hardware pipeline to complete the decoding processing. The

driver specifies through MFD commands of what to send out from the BSD unit and where to send the BSD

output.

a. For transcoding (including transrating and transcaling), part of the BSD output (a series of per-

MB record) can be sent to memory for further processing to encode into a difference output

format. This function is named as StreamOut. When StreamOut is active, not all MB information

needs to be sent, only MVs and selective MB coding information.

2. IT Mode: In this mode, the BSD is not invoked. Instead host performs all the bitstream decoding and parsing;

and the result are saved into memory in a specific per-MB record format. The MFD Engine VCS reads in

these records one at time and finish the rest of the decoding (IT, MC, IntraPred and ILDB).

MB information is organized into two indirect data buffers, one for MVs and one for residue coefficients.

As such, two indirect base address pointers are defined.

MFX_SURFACE_STATE

MFX_PIPE_BUF_ADDR_STATE

MFX_IND_OBJ_BASE_ADDR_STATE

MFX_BSP_BUF_BASE_ADDR_STATE

MFX_PAK_INSERT_OBJECT

MFX_STITCH_OBJECT

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MFX_QM_STATE

Bits

31:24

23:16

15:8

7:0

Dword 1

QuantMatrix[0][3]

QuantMatrix[0][2]

QuantMatrix[0][1]

QuantMatrix[0][0]

Dword 2

QuantMatrix[0][7]

QuantMatrix[0][6]

QuantMatrix[0][5]

QuantMatrix[0][4]

Dword 3

QuantMatrix[1][3]

QuantMatrix[1][2]

QuantMatrix[1][1]

QuantMatrix[1][0]

…

Dword 16

QuantMatrix[7][7]

QuantMatrix[7][6]

QuantMatrix[7][5]

QuantMatrix[7][4]

MFX_FQM_STATE

This is a frame-level state. Reciprocal Scaling Lists are always sent from the driver regardless whether they are

specified by an application or the default/flat lists are being used. This is done to save the ROM (to store the default

matrices) inside the PAK Subsystem. Hence, the driver is responsible for determining the final set of scaling lists to

be used for encoding the current slice, based on the AVC Spec (Fall-Back Rules A and B). For encoding, there is no

need to send the qm_list_flags[i], i=0 to7 and qm_present_flag to the PAK, since Scaling Lists syntax elements are

encoded above Slice Data Layer.

FQM Reciprocal Scaling Lists elements are 16-bit each, conceptually equal to 1/ScaleValue. QM matrix elements are

8-bit each, equal to ScaleValue. However, in AVC spec., the Reciprocal Scaling Lists elements are not exactly equal

to one-over of the corresponding Scaling Lists elements. The numbers are adjusted to simplify hardware

implementation.

For all the description below, a scaling list set contains 6 4x4 scaling lists (or forward scaling lists) and 2 8x8 scaling

lists (or forward scaling lists).

In MFX PAK mode, PAK needs both forward Q scaling lists and IQ scaling lists. The IQ scaling lists are sent as in

MFD in raster scan order as shown in MFX_AVC_QM_STATE. But the Forward Q scaling lists are sent in transport

form, i.e. column-wise raster order (column-by-column) to simplify the H/W.

Precisely, if the reciprocal forward scaling matrix is F[4][4], then the 16 word of the matrix will be set as

the following:

For JPEG encoder, 16-bit precision is used for each element 1/QM matrix. The 32 DWords are used for 64 QM

elements with the following data structure:

Bits 15:0

Bits 31:16

DWord1

1/QM[0][0]

1/QM[1][0]

DWord2

1/QM[2][0]

1/QM[3][0]

DWord3

1/QM[4][0]

1/QM[5][0]

DWord4

1/QM[6][0]

1/QM[7][0]

DWord5

1/QM[0][1]

1/QM[1][1]

DWord6

1/QM[2][1]

1/QM[3][1]

DWord7

1/QM[4][1]

1/QM[5][1]

DWord8

1/QM[6][1]

1/QM[7][1]

…

DWord31

1/QM[4][7]

1/QM[5][7]

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Bits 15:0

Bits 31:16

DWord32

1/QM[6][7]

1/QM[7][7]

Bitplane Buffer

Bitplane coding is used in seven different cases in VC-1, although not all the seven syntax elements are

present in the same picture header at the same time. The following list shows which syntax elements are

coded as bitplanes in each picture header:

Progressive I and BI picture headers in AP: ACPRED, OVERFLAGS

Field interlace I and BI picture headers in AP: ACPRED, OVERFLAGS

Frame interlace I and BI picture headers in AP: FIELDTX, ACPRED, OVERFLAGS

Frame interlace P picture headers in AP: SKIPMB

Progressive P picture headers in SP and MP: MVTYPEMB, SKIPMB

Progressive P picture headers in AP: MVTYPEMB, SKIPMB

Field interlace B picture headers in AP: FORWARDMB

Frame interlace B picture headers in AP: DIRECTMB, SKIPMB

Progressive B picture headers in AP: DIRECTMB, SKIPMB

Progressive B picture headers in MP: DIRECTMB, SKIPMB

There are also seven different modes of coding the bitplane information. Except when the bitplane is

coded in raw mode, the bitplane is decoded by the host and provided to the hardware in the bitplane

buffer.

Since at most three bitplanes are encoded in any picture header, instead of using a complete byte for

signaling the values of all the seven possible bitplanes for each MB, a more efficient approach is used

with each byte divided in two nibbles and each nibble carries the data of up to four bitplanes for one MB.

PictureType

Bits 3, 7

Bit 2, 6

Bits 1, 5

Bits 0, 4

I or BI

OVERFLAGS

ACPRED

FIELDTX

MVTYPEMB

SKIPMB

FORWARDMB

SKIPMB

DIRECTMB

The bytes containing the above defined nibbles are stored in the bitplane buffer in raster scan order. The

bitplane buffer is a linear buffer with a buffer pitch (as defined by Bitplane Buffer Pitch field in

VC1_BSD_PIC_STATE command). When the number of macroblock in a row is odd, the last byte of the

row containing the last macroblock in bits 0-3. The first macroblock of the next row starts at the next

pitch offset from the first macroblock of the current row.

The bitplane buffer structure must be sent once per picture only if there is one or more syntax elements

coded as bitplanes in the picture header.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Video Codecs

The following sections contain the various registers for video codec support. Specifically, the codec types

supported are:

Supported Codec Types

Advanced Video Coding (AVC)/ H.264/MPEG-4 Part 10 (MVC)

MPEG-2 (H.222/H.262) — Used in Digital Video Broadcast and DVDs

VC1 — SMPTE 421M, known informally as VC-1

VP8 — Video compression format

JPEG and MJPEG — A video format in which video gram or interlaced field of a digital video sequence is

compressed separately as a JPEG image

Other Codec Functions

Internal Media Rowstore table – An internal Media Rowstore Storage is added to reduce memory

read/write to save power. If the internal Media Rowstore exists, driver should use the storage as the

following table indicates.

AVC/VC1/MPEG2/JPEG/VP8 Decoder/Encoder:

[BSD is bitstream decoder rowstore; MPR is Motion Prediction rowstore; IP is Intra Prediction rowstore; VLF is loop

filter rowstore]

Codec

Mode

Frame

Width

BSD

MPR

VLF

VDE

BSD

Addr

MPR

Addr

VLF

ADDR

VDE

ADDR

AVC Dec

Frame/Field

< 2k

128

256

384

N/A

2k -> 3k

192

384

N/A

3k -> 4k

256

N/A

Mbaff

< 2k

256

512

N/A

2k -> 4k

N/A

AVC Enc

Frame/Field

< 2k

N/A

256

384

N/A

2k -> 3k

N/A

384

N/A

3k -> 4k

N/A

256

N/A

Mbaff

< 2k

N/A

512

N/A

2k -> 4k

N/A

JPEG

Dec/Enc

N/A

VP8 Dec

Frame

< 2k

N/A

256

384

N/A

2k -> 3k

N/A

384

N/A

3k -> 4k

N/A

256

N/A

VP8 Enc

Frame

N/A

MPEG2

N/A

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Codec

Mode

Frame

Width

BSD

MPR

VLF

VDE

BSD

Addr

MPR

Addr

VLF

ADDR

VDE

ADDR

VC1 Dec

N/A

AVC (H.264)

AVC Common Commands

MFX Commands are organized into groups based on their scope of functioning. There are Pipeline

Common state commands that are common to all codecs (encoder and decoder) and is applicable to the

processing of one full frame/field. There are also individual codec Common state commands that are

common to both encoder and decoder of that particular codec. These latter common state commands,

some are applicable at the processing of one full frame/field, and some are applicable at the processing

of an individual slice level.

MFX_AVC_IMG_STATE

A new command is added to support MPEG transport stream encapsulation of AVC bitstream in Encoder

mode. This command should be used only when MPEG transport stream is needed.

MFX_AVC_DIRECTMODE_STATE

MFX_AVC_SLICE_STATE

MFX_AVC_REF_IDX_STATE

MFX_AVC_WEIGHTOFFSET_STATE

AVC Decoder Commands

These are decoder-only commands. They provide the pointer to the compressed input bitstream to be

decoded.

MFD_AVC_DPB_STATE

NOTE modified from DXVA2 – The values in RefFrameList and UsedForReference_Flag are the primary

means by which the H/W can determine whether the corresponding entries in RefFrameList, POCList,

LTSTFrameNumList, and Non-ExistingFrame_Flag should be considered valid for use in the decoding

process of the current picture or not. When RefFrameList[i] is marked to be invalid, the values of

POCList[i][0], POCList[i][1], LTSTFrameNumList[i], UsedForReference_Flag[i], and Non-

ExistingFrame_Flag[i] must all be equal to 0. When UsedForReference_Flag[i] = 0, the value of

RefFrameList[i] must be marked invalid.

MFD_AVC_SLICEADDR

MFD_AVC_BSD_OBJECT

Inline Data Description for MFD_AVC_BSD_Object

MFD_AVC_PICID_STATE

NOTE 1: In AVC short format, PictureIDList has one-to-one corresponding to LongTermFrame_Flag list,

Non-ExistingFrame_flag list, UsedForReference_Flag list, FrameNumList list in MFD_AVC_DPB_STATE.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

NOTE 2: PictureIDList is only used to identify reference picture across frames. Hardware will convert the

picture in the RefFrameList to PictureID before writing out DMV data and convert back to RefFrameList

Index after reading out DMV data. The reference pictures and their orders in the RefFrameList can be

changed across frames.

Session Decoder StreamOut Data Structure

When StreamOut is enabled, per MB intermediated decoded data (MVs, mb_type, MB qp, etc.) are sent

to the memory in a fixed record format (and of fixed size). The per-MB records must be written in a strict

raster order and with no gap (i.e. every MB regardless of its mb_type and slice type, must have an entry

in the StreamOut buffer). Therefore, the consumer of the StreamOut data can offset into the StreamOut

Buffer (StreamOut Data Destination Base Address) using individual MB addresses.

A StreamOut Data record format is detailed as follows:

DWord

Bit

Description

Reserved MBZ

22-20

EdgeFilterFlag (AVC) / OverlapSmoothFilter (VC1)

19:17

CodedPatternDC (for AVC only, 111b for others)

The field indicates whether DC coefficients are sent.

1 bit each for Y, U and V.

Reserved MBZ

Transform8x8Flag

When it is set to 0, the current MB uses 4x4 transform. When it is set to 1, the current MB uses 8x8

transform. The transform_szie_8x8_flag syntax element, if present in the output bitstream, is the

same as this field. However, whether transform_szie_8x8_flag is present or not in the output

bitstream depends on several conditions:

This field is only allowed to be set to 1 for two conditions:

It must be 1 if IntraMbFlag = INTRA and IntraMbMode = INTRA_8x8

It may be 1 if IntraMbFlag = INTER and there is no sub partition size less than 8x8

Otherwise, this field must be set to 0.

0: 4x4 integer transform

1: 8x8 integer transform

MbFieldFlag

This field specifies whether current macroblock is coded as a field or frame macroblock in MBAFF

mode.

This field is exactly the same as FIELD_PIC_FLAG syntax element in non-MBAFF mode.

Same as the mb_field_decoding_flag syntax element in AVC spec.

0 = Frame macroblock

1 = Field macroblock

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

IntraMbFlag

This field specifies whether the current macroblock is an Intra (I) macroblock.

I_PCM is considered as Intra MB.

For I-picture MB (IntraPicFlag =1), this field must be set to 1.

This flag must be set in consistent with the interpretation of MbType (inter or intra modes).

0: INTER (inter macroblock)

1: INTRA (intra macroblock)

12:8

MbType5Bits

This field is encoded to match with the best macroblock mode determined as described in the

next section. It follows AVC encoding for inter and intra macroblocks.

MbPolarity

FieldMB Polarity - vctrl_vld_top_field AVC

Reserved MBZ

5:4

IntraMbMode

This field is provided to carry information partially overlapped with MbType.

This field is only valid if IntraMbFlag = INTRA, otherwise, it is ignored by hardware.

Reserved MBZ

MbSkipFlag

Reserved MBZ (DXVA Encoder). HW (VDSunit) doesn’t have skip MB info.

It sets to 1 if any of the sub-blocks is inter, uses predicted MVs, and skips sending MVs explicitly in

the code stream. Currently H/W can provide this flag and is defaulted to 0 always.

1:0

InterMbMode

This field is provided to carry redundant information as that in MbType. It also carries additional

information such as skip.

This field is only valid if IntraMbFlag =INTER, otherwise, it is ignored by hardware.

31:16

MbYCnt (Vertical Origin).

This field specifies the vertical origin of current macroblock in the destination picture in units of

macroblocks.

Format = U8 in unit of macroblock.

15:0

MbXCnt (Horizontal Origin).

This field specifies the horizontal origin of current macroblock in the destination picture in units of

macroblocks.

Format = U8 in unit of macroblock.

Conceal MB Flag.

This field specifies if the current MB is a conceal MB, use in AVC/VC1/MPEG2 mode.

Last MB of the Slice Flag.

This field indicate the current MB is a last MB of the slice. Use in AVC/VC1/MPEG2 mode.

29:24

Reserved

23:20

CbpAcV

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

0 in a bit – indicates the corresponding 8x8 block or 4x4 sub-block is not present (because all

coefficient values are zero)

1 in a bit – indicates the corresponding 8x8 block or 4x4 sub-block is present (although it is still

possible to have all its coefficients be zero – bad coding).

19:16

CbpAcU

0 in a bit – indicates the corresponding 8x8 block or 4x4 sub-block is not present (because all

coefficient values are zero)

1 in a bit – indicates the corresponding 8x8 block or 4x4 sub-block is present (although it is still

possible to have all its coefficients be zero – bad coding).

15:0

CbpAcY

0 in a bit – indicates the corresponding 8x8 block or 4x4 sub-block is not present (because all

coefficient values are zero)

1 in a bit – indicates the corresponding 8x8 block or 4x4 sub-block is present (although it is still

possible to have all its coefficients be zero – bad coding).

Bit15=Y0Sub0, Bit0=Y3Sub3

31:28

AVC

Skip8x8Pattern (AVC)

This field indicates whether each of the four 8x8 sub macroblocks is using the predicted MVs and

will not be explicitly coded in the bitstream (the sub macroblock will be coded as direct mode). It

contains four 1-bit subfields, corresponding to the 4 sub macroblocks in sequential order. The

whole macroblock may be actually coded as B_Direct_16x16 or B_Skip, according to the

macroblock type conversion rules described in a later sub section.

This field is only valid for a B slice. It is ignored by hardware for a P slice. Hardware also ignores this

field for an intra macroblock.

0 in a bit – Corresponding MVs are sent in the bitstream

1 in a bit – Corresponding MVs are not sent in the bitstream

27:25

Reserved

24:16

NzCoefCountMB – all coded coefficients input including AC/DC blocks in current MB.

Range 0 to 384 (9 bits)

15:8

MbClock16 – MB compute clocks in 16-clock unit.

mbz (AVC) / QScaleType (MPEG2)

6:0

QpPrimeY (AVC) / QScaleCode (MPEG2)

The luma quantization index. This is the per-MB QP value specified for the current MB.

4 to 6

31:0

Each

For intra macroblocks, definition of these fields are specified in 1

For inter macroblocks, definition of these fields are specified in 2

31:24

Reserved

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

23:20

MvFieldSelect (Ref polarity top or bottom bits) for VC1 and MPEG2

vcp_vds_mvdataR[162:159]

VC1

vmd_vds_mt_vert_fld_selR[3:0]

MPEG2

19:12

Reserved

11:10

SubBlockCodeType V

(If 8x8, 8x4, 4x8, 4x4 type)

9:8

SubBlockCodeType U

(specifies 8x8, 8x4, 4x8, 4x4 type) VC1

7:6

SubBlockCodeType Y3

(specifies 8x8, 8x4, 4x8, 4x4 type) VC1

5:4

SubBlockCodeType Y2

(specifies 8x8, 8x4, 4x8, 4x4 type) VC1

3:2

SubBlockCodeType Y1

(specifies 8x8, 8x4, 4x8, 4x4 type) VC1

1:0

SubBlockCodeType Y0

(specifies 8x8, 8x4, 4x8, 4x4 type) VC1

Inter Cases:

31:16

MvFwd[0].y – y-component of the forward motion vector of the 1

8x8 or 1

4x4 subblock

15:0

MvFwd[0].x – x-component of the forward motion vector of the 1

8x8 or 1

4x4 subblock

31:0

MvBck[0] – the backward motion vector of the 1

8x8 or 1

4x4 subblock

31:0

MvFwd[1] – the forward motion vector of the 2

8x8 or 4

4x4 subblock

31:0

MvBck[1] – the backward motion vector of the 2

8x8 or 4

4x4 subblock

31:0

MvFwd[2] – the forward motion vector of the 3

8x8 or 8

4x4 subblock

31:0

MvBck[2] – the backward motion vector of the 3

8x8 or 8

4x4 subblock

31:0

MvFwd[3] – the forward motion vector of the 4

8x8 or 12

4x4 subblock

31:0

MvBck[3] – the backward motion vector of the 4th 8x8 or 12

4x4 subblock

Intra Cases:

8 to 15

31:0

Reserved MBZ

The inline data content of Dwords 4 to 6 is defined either for intra prediction or for inter prediction, but

not both.

Inline data subfields for an Intra Macroblock

DWord

Bit

Description

31:16

LumaIndraPredModes[1]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block of a MB, 4-bit each.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

AVC: See the bit assignment table later in this section.

VC1: MBZ.

MPEG2: MBZ.

15:0

LumaIndraPredModes[0]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block, four 8x8 block or one intra16x16

of a MB.

4-bit per 4x4 sub-block (Transform8x8Flag=0, Mbtype=0 and intraMbFlag=1) or 8x8 block

(Transform8x8Flag=1, Mbtype=0, MbFlag=1), since there are 9 intra modes.

4-bit for intra16x16 MB (Transform8x8Flag=0, Mbtype=1 to 24 and intraMbFlag=1), but only the

LSBit[1:0] is valid, since there are only 4 intra modes.

AVC: See the bit assignment table later in this section.

VC1: MBZ.

MPEG2: MBZ.

AVC

INTRA

31:16

LumaIndraPredModes[3]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block of a MB, 4-bit each.

AVC: See the bit assignment table later in this section.

VC1: MBZ.

MPEG2: MBZ.

15:0

LumaIndraPredModes[2]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block of a MB, 4-bit each.

AVC: See the bit assignment later in this section.

VC1: MBZ.

MPEG2: MBZ.

31:8

Reserved (Reserved for encoder turbo mode IntraResidueDataSize, when this is not 0, optional

residue data are provided to the PAK; Reserved for decoder)

7:0

MbIntraStruct

The IntraPredAvailFlags[4:0] have already included the effect of the constrained_intra_pred_flag. See

the diagram later for the definition of neighbors position around the current MB or MB pair (in

MBAFF mode).

1 – IntraPredAvailFlagX, indicates the values of samples of neighbor X can be used in intra

prediction for the current MB.

0 – IntraPredAvailFlagX, indicates the values of samples of neighbor X is not available for intra

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

prediction of the current MB.

IntraPredAvailFlag-A and -E can only be different from each other when constrained_intra_pred_flag

is equal to 1 and mb_field_decoding_flag is equal to 1 and the value of the mb_field_decoding_flag

for the macroblock pair to the left of the current macroblock is equal to 0 (which can only occur

when MbaffFrameFlag is equal to 1).

IntraPredAvailFlag-F is used only if

o it is in MBAFF mode, i.e. MbaffFrameFlag = 1,

o the current macroblock is of frame type, i.e. MbFieldFag = 0, and

o the current macroblock type is Intra8x8,

that is, IntraMbFlag = INTRA, IntraMbMode = INTRA_8x8, and Transform8x8Flag = 1.

In any other cases IntraPredAvailFlag-A shall be used instead.

Bits

IntraPredAvailFlags[4:0] Definition

IntraPredAvailFlagF – F (Left 8

row (-1,7) neighbor)

IntraPredAvailFlagA – A

(Left neighbor top half)

IntraPredAvailFlagE – E

(Left neighbor bottom half)

IntraPredAvailFlagB – B

(Top neighbor)

IntraPredAvailFlagC – C

(Top right neighbor)

IntraPredAvailFlagD – D

(Top left corner neighbor)

1:0

ChromaIntraPredMode –

2 bits to specify 1 of 4 chroma intra prediction mode, see the table in later section.

Inline data subfields for an Inter Macroblock

DWord

Bit

Description

31:24

Reserved: MBZ (DXVA Decoder)

23:16

Reserved: MBZ (DXVA Decoder)

15:8

SubMbPredModes[bit 7:0] (Sub Macroblock Prediction Mode)

This field describes the prediction mode of the sub macroblocks (four 8x8 blocks). It contains four

subfields each with 2-bits, corresponding to the 4 fixed size 8x8 sub macroblocks in sequential

order.

This field is provided for MB with sub_mb_type equal to BP_8x8 only (B_8x8 and P_8x8 as defined in

DXVA)

This field is derived from MbType for a non-BP_8x8 inter macroblock, and carries redundant

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

information as MbType)

Bits [1:0]: SubMbPredMode[0] – for 8x8 Block 0

Bits [3:2]: SubMbPredMode[1] – for 8x8 Block 1

Bits [5:4]: SubMbPredMode[2] – for 8x8 Block 2

Bits [7:6]: SubMbPredMode[3] – for 8x8 Block 3

Blocks of the MB is numbered as follows :

0 1

2 3

Each 2-bit value [1:0] is defined as :

00 – Pred_L0

01 – Pred_L1

10 – BiPred

For VC1:

Bits [1:0]: “00”= Y0 Forward only, “01”= Y0 Backward only, “10”= Y0 Bi direction

Bits [3:2]: SubMbPredMode[1] – for 8x8 Block 1

Bits [5:4]: SubMbPredMode[2] – for 8x8 Block 2

Bits [7:6]: SubMbPredMode[3] – for 8x8 Block 3

7:0

SubMbShape[bit 7:0] (Sub Macroblock Shape)

This field describes the sub-block partitioning of each sub macroblocks (four 8x8 blocks). It contains

four subfields each with 2-bits, corresponding to the 4 fixed size 8x8 sub macroblocks in sequential

order.

This field is provided for MB with sub_mb_type equal to BP_8x8 only (B_8x8 and P_8x8 as defined in

DXVA)

This field is forced to 0 for a non-BP_8x8 inter macroblock, and effectively carries redundant

information as MbType).

Bits [1:0]: SubMbShape[0] – for 8x8 Block 0

Bits [3:2]: SubMbShape[1] – for 8x8 Block 1

Bits [5:4]: SubMbShape[2] – for 8x8 Block 2

Bits [7:6]: SubMbShape[3] – for 8x8 Block 3

Blocks of the MB is numbered as follows :

0 1

2 3

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

Each 2-bit value [1:0] is defined as :

00 – SubMbPartWidth=8, SubMbPartHeight=8

01 – SubMbPartWidth=8, SubMbPartHeight=4

10 – SubMbPartWidth=4, SubMbPartHeight=8

11 – SubMbPartWidth=4, SubMbPartHeight=4

For VC-1, This field indicates the transformation types used for luma components, 2 bits for each

8x8.

31:24

Frame Store ID L0[3]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation)

23:16

Frame Store ID L0[2]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation)

15:8

Frame Store ID L0[1]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation).

7:0

Frame Store ID L0[0]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation)

31:24

Frame Store ID L1[3]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation)

23:16

Frame Store ID L1[2]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

implementation)

15:8

Frame Store ID L1[1]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation)

7:0

Frame Store ID L1[0]

Support up to 4 Frame store ID per L0 direction, one per MB partition, if exists. See details in later

section. This field specifies the frame Store ID into the Reference Picture List0 Table.

Bit 7: Must Be One: (This is reserved for control fields in future extension, when reference index are

generated instead of frame store ID)

1: indicate it is in Frame store ID format.

0: indicate it is in Reference Index format.

Bit 6:5: reserved MBZ

Bit 4:0: Frame store index or Frame Store ID (Bit 4:1 is used to form the binding table index in intel

implementation)

AVC Encoder PAK Commands

Each PAK Commands is composed of a command op-code DW and one or more command data DWs

(inline data). The size of each command is specified as part of the op-code DW. Most of the commands

have fixed size, except some are allowed to be of variable length.

There is an inherent order of executing MFC PAK commands that driver must follow.

MFC_AVC_PAK_OBJECT

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

PAK Object Inline Data Description

The Inline Data includes all the required MB encoding states, constitute part of the Slice Data syntax

elements, MB Header syntax elements and their derivatives. It provides information for the following

operations:

1. Forward and Inverse Transform

2. Forward and Inverse Quantization

3. Advanced Rate Control (QRC)

4. MB Parameter Construction (MPC)

5. CABAC/CAVLC encoding

6. Bit stream packing

7. Intra and inter-Prediction decoding loop

8. Internal error handling

These state/parameter values may subject to change on a per-MB basis, and must be provided in each

MFC_AVC_PAK_OBJECT command. The values set for these variables are retained internally, until they are

reset by hardware Asynchronous Reset or changed by the next MFC_AVC_PAK_OBJECT command.

The inline data has been designed to match the DXVA 2.0, with the exception of the starting byte

(DW0:0-7) and the ending dword (DW7:0-31).

The Deblocker Filter Control flags (FilterInternalEdgesFlag, FilterTopMbEdgeFlag and

FilterLeftMbEdgesFlag) are generated by H/W, which are depending on MbaffFrameFlag, CurrMbAddr,

PicWidthInMbs and disable_deblocking_filter_idc states.

Current MB [x,y] address is not sent, it is assumed that the H/W will keep track of the MB count and

current MB position internally.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

Reserved: MBZ

CbpDcY. This field specifies if the Luma DC sub-block is coded. Setting it to 0 will force PAK to zero

out the Luma sub-block. Otherwise, whether the sub-block is coded will be determined by the

quantization process.

1 – the 4x4 DC-only Luma sub-block of the Intra16x16 coded MB is present; it is still possible that all

DC coefficients are zero.

0 – no 4x4 DC-only Luma sub-block is present; either not in Intra16x16 MB mode or all DC

coefficients are zero.

Programming Note

Context:

PAK Object Inline Data Description - CbpDcY

When Reference Mb: IPCM or inferred IPCM, current mb: base mode flag = 1: TcoeffLvlPredFlag =

1 ; all bits in CbpDcY, CbpDcU, CbpDcV, Cbp4x4Y[15:0], Cbp4x4V[15:0] and Cbp4x4U[15:0] must

set to 1’s.

CbpDcU. This field specifies if the Chroma Cb DC sub-block is coded. Setting it to 0 will force PAK

to zero out the Luma sub-block. Otherwise, whether the sub-block is coded will be determined by

the quantization process.

1 – the 2x2 DC-only Chroma Cb sub-block of all coded MB (any type) is present; it is still possible

that all DC coefficients are zero.

0 – no 2x2 DC-only Chroma Cb sub-block is present; all DC coefficients are zero.

Programming Note

Context:

PAK Object Inline Data Description

When Reference Mb: IPCM or inferred IPCM, current mb: base mode flag = 1: TcoeffLvlPredFlag =

1 ; all bits in CbpDcY, CbpDcU, CbpDcV, Cbp4x4Y[15:0], Cbp4x4V[15:0] and Cbp4x4U[15:0] must

set to 1’s.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

CbpDcV. This field specifies if the Chroma Cb DC sub-block is coded. Setting it to 0 will force PAK

to zero out the Luma sub-block. Otherwise, whether the sub-block is coded will be determined by

the quantization process.

1 – the 2x2 DC-only Chroma Cr sub-block of all coded MB (any type) is present; it is still possible

that all DC coefficients are zero.

0 – no 2x2 DC-only Chroma Cr sub-block is present; all DC coefficients are zero.

Programming Note

Context:

PAK Object Inline Data Description

When Reference Mb: IPCM or inferred IPCM, current mb: base mode flag = 1: TcoeffLvlPredFlag =

1 ; all bits in CbpDcY, CbpDcU, CbpDcV, Cbp4x4Y[15:0], Cbp4x4V[15:0] and Cbp4x4U[15:0] must

set to 1’s.

Reserved: MBZ

(reserved for future use as ExternalResidBufFlag for turbo mode)

Transform8x8Flag

This field indicates that 8x8 transform is used for the macroblock.

When it is set to 0, the current MB uses 4x4 transform. When it is set to 1, the current MB uses 8x8

transform. The transform_size_8x8_flag syntax element, if present in the output bitstream, is the

same as this field. However, whether transform_szie_8x8_flag is present or not in the output

bitstream depends on several other conditions.

This field is only allowed to be set to 1 for two conditions:

It must be 1 if IntraMbFlag = INTRA and IntraMbMode = INTRA_8x8

It may be 1 if IntraMbFlag = INTER and there is no sub partition size less than 8x8

Otherwise, this field must be set to 0.

Programming Note

Context:

PAK Inline Object Data Description

When AvcImgState EntropyCodingFlag is 1(CABAC), this field cannot be 1.

0: 4x4 integer transform

1: 8x8 integer transform

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

FieldMbFlag

This field specifies the field polarity of the current macroblock, as the mb_field_decoding_flag syntax

element in AVC spec.

This field specifies whether current macroblock is coded as a field or frame macroblock in MBAFF

mode. It is exactly the same as FIELD_PIC_FLAG syntax element in non-MBAFF mode.

0 = Frame macroblock

1 = Field macroblock

IntraMbFlag

This field specifies whether the current macroblock is an Intra (I) macroblock. I_PCM is considered as

Intra MB.

For I-picture MB (IntraPicFlag =1), this field must be set to 1.

This flag must be set in consistent with the interpretation of MbType (inter or intra modes).

0: INTER (inter macroblock)

1: INTRA (intra macroblock)

12:8

MbType5Bits

This field is encoded to match with the best macroblock mode determined as described in the next

section. It follows an unified encoding for inter and intra macroblocks according to AVC Spec.

FieldMbPolarityFlag

This field indicates the field polarity of the current macroblock.

Within an MbAff frame picture, this field may be different per macroblock and is set to 1 only for

the second macroblock in a MbAff pair if FieldMbFlag is set. Otherwise, it is set to 0.

Within a field picture, this field is set to 1 if the current picture is the bottom field picture.

Otherwise, it is set to 0. It is a constant for the whole field picture.

This field is reserved and MBZ for a progressive frame picture.

0 = Current macroblock is a field macroblock from the top field

1 = Current macroblock is a field macroblock from the bottom field

Programming Note: Here bits [26:24] (MbAffFieldFlag and FiedlMbPolarityFlag) match with bits [10:8]

of the Media Block Read message descriptor, simplifying the programming for message generation, as

when MbAffFieldFlag is "1", kernels need to override the original "frame" surface state set for MBAFF

frame picture.

MB Reserved: Inter MB converted to IPCM.

This field is for HW purposes only.

SW should not use it.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

5:4

IntraMbMode

This field is provided to carry information partially overlapped with MbType.

This field is only valid if IntraMbFlag = INTRA, otherwise, it is ignored by hardware..

Reserved: MBZ

SkipMbFlag

By setting it to 1, this field forces an inter macroblock to be encoded as a skipped macroblock. It is

equivalent to mb_skip_flag in AVS spec, indicating that a macroblock is inferred as a P_Skip (or

B_Skip) in a P Slice (or B Slice). Hardware honors input MVs for motion prediction and forces CBP to

zero.

By setting it to 0, an inter macroblock will be coded as a normal inter macroblock. The macroblock

may still be coded as a skipped macroblock, according to the macroblock type conversion rules

described in the later sub sections.

This field can only be set to 1 for certain values of MbType. See details later.

This field is only valid for an inter macroblock. For intra MB (bit 13 of this DW set to one), this bit

must be set to zero.

0 = not a skipped macroblock

1 = is coded as a skipped macroblock

1:0

InterMbMode

This field is provided to carry redundant information as that encoded in MbType.

This field is only valid if IntraMbFlag =0, otherwise, it is ignored by hardware.

15:8

MbYCnt (Vertical Origin). This field specifies the vertical origin of current macroblock in the

destination picture in units of macroblocks.

Format = U8 in unit of macroblock.

7:0

MbXCnt (Horizontal Origin). This field specifies the horizontal origin of current macroblock in the

destination picture in units of macroblocks.

Format = U8 in unit of macroblock.

31:16

Cbp4x4V (Coded Block Pattern Cr)

Only the lower 4 bits [3:0] are valid for 4:2:0. The 4x4 Cr sub-blocks are numbered as

blk0 1 bit3 2

blk2 3 bit1 0

The cbpCr bit assignment is cbpCr bit [3 - X] for sub-block_num X.

0 in a bit - indicates the corresponding 4x4 sub-block is not present (because all coefficient values

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

are zero), or force to zero for PAK.

1 in a bit - indicates the corresponding 4x4 sub-block is present (although it is still possible to have

all its coefficients be zero - bad coding).

For monochrome, this field is ignored.

For 4.2.2, [23:16] for U(Cb), and [31:24] ignored.

For 4.4.4, the definition is the same as for luma component: 1bit per 4x4 block.

Programming Note

Context:

PAK Object Inline Data Description

When Reference Mb: IPCM or inferred IPCM, current mb: base mode flag = 1: TcoeffLvlPredFlag =

1 ; all bits in CbpDcY, CbpDcU, CbpDcV, Cbp4x4Y[15:0], Cbp4x4V[15:0] and Cbp4x4U[15:0] must

set to 1’s.

15:0

Cbp4x4U (Coded Block Pattern Cb)

Only the lower 4 bits [3:0] are valid for 4:2:0. The 4x4 Cb sub-blocks are numbered as

blk0 1 bit3 2

blk2 3 bit1 0

The cbpCb bit assignment is cbpCb bit [3 - X] for sub-block_num X.

0 in a bit - indicates the corresponding 4x4 sub-block is not present (because all coefficient values

are zero), or force to zero for PAK.

1 in a bit - indicates the corresponding 4x4 sub-block is present (although it is still possible to have

all its coefficients be zero - bad coding).

For monochrome, this field is ignored.

For 4.2.2, [7:0] for U(Cb), and [15:8] ignored.

For 4.4.4, the definition is the same as for luma component: 1bit per 4x4 block.

Programming Note

Context:

PAK Object Inline Data Description

When Reference Mb: IPCM or inferred IPCM, current mb: base mode flag = 1: TcoeffLvlPredFlag =

1 ; all bits in CbpDcY, CbpDcU, CbpDcV, Cbp4x4Y[15:0], Cbp4x4V[15:0] and Cbp4x4U[15:0] must

set to 1’s.

31:28

Skip8x8Pattern

This field indicates whether each of the four 8x8 sub macroblocks is using the predicted MVs and

will not be explicitly coded in the bitstream (the sub macroblock will be coded as direct mode). It

contains four 1-bit subfields, corresponding to the 4 sub macroblocks in sequential order. The

whole macroblock may be actually coded as B_Direct_16x16 or B_Skip, according to the macroblock

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

type conversion rules described in a later sub section.

This field is only valid for a B slice. It is ignored by hardware for a P slice. Hardware also ignores this

field for an intra macroblock.

0 in a bit – Corresponding MVs are sent in the bitstream

1 in a bit – Corresponding MVs are not sent in the bitstream

EnableCoeffClamp

1 = the magnitude of coefficients of the current MB will be clamped based on the clamping matrix

after quantization

0 = no clamping

LastMbFlag

1 – the current MB is the last MB in the current Slice

0 – the current MB is not the last MB in the current Slice - Reserved MBZ.

SkipMbConvDisable

This is a per-MB level control to enable and disable skip conversion. This field is ORed with

SkipConvDisable field. This field is only valid for a P or B slice. It must be zero for other slice types.

Rules are provided in Section Macroblock Type Conversion Rules

0 - Enable skip type conversion for the current macroblock

1 - Disable skip type conversion for the current macroblock

Reserved MBZ.

23:16

Reserved. Ignored by HW, this field will be re-derived internally.

(was QpPrimeV. For 8-bit pixel data, QpCr is the same as QpPrimeCr, and it takes on a value in the

range of 0 to 51, positive integer.)

15:8

Reserved. Ignored by HW, this field will be re-derived internally.

(Was QpPrimeU. For 8-bit pixel data, QpCb is the same as QpPrimeCb, and it takes on a value in

the range of 0 to 51, positive integer.)

7:0

QpPrimeY

This is the per-MB QP value specified for the current MB.

For 8-bit pixel data, QpY is the same as QpPrimeY, and it takes on a value in the range of 0 to 51,

positive integer.

Programming Note

Context:

PAK Object Inline Data Description

This value may differ from the actual codes, when HW QRC is on

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

7 .. 9

31:0

Each

For intra macroblocks, definition of these fields are specified in PAK Object Inline Data Description

For inter macroblocks, definition of these fields are specified in PAK Object Inline Data Description

31:24

MaxSizeInWord

PAK should not exceed this budget accumulatively, otherwise it will trickle the PANIC mode.

23:16

TargetSizeInWord

PAK should use this budget accumulatively to decide if it needs to limit the number of non-zero

coefficients.

15:0

Reserved: MBZ

15:0

Lambda_Or_RoundingOffset

When TQEnb=1, this 16-bit unsigned value multiplied by 2 is used as a lambda for the rate-

distortion cost estimation in Trellis quantization (TQ). If the upper 4 bits are all set to 1 (0xFXXX), TQ

is disabled and the regular quantizer is used. Thus, the valid range is 0~0xEFFF. When TQ is enabled

per MB, the TQR in MFC_AVC_IMG_STATE is used for rounding quantization coefficients.

When TQEnb=0 or the upper 4 bits are all set to 1, the lower 4-bit value indicates the rounding

precision (offset) for the regular quantizer. The values ranging 0001b ~ 0111 are reserved.

Value

Name

0000b

RoundInterEnable, RoundInter, RoundIntraEnable, and RoundIntra defined in

MFC_AVC_SLICE_STATE are used as rounding precision.

1000b

+1/16

1001b

+2/16

1010b

+3/16

1011b

+4/16

1100b

+5/16

1101b

+6/16

1110b

+7/16

1111b

+8/16

Format: U16

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Inline data for LumaIntraMode

ExtendedForm

0 or 1

Intra4x4

Intra8x8

Intra16x16

Intra8x8

Intra16x16

DW4 – 31:28

Block 7

Block 0

DW4 – 27:24

Block 6

Block 0

DW4 – 23:20

Block 5

Block 0

DW4 – 19:16

Block 4

Block 0

DW4– 15:12

Block 3

Block 0

DW4 – 11:8

Block 2

Block 0

DW4 – 7:4

Block 1

Block 0

DW4 – 3:0

Block 0

DW5 – 31:28

Block 15

Block 0

DW5 – 27:24

Block 14

Block 0

DW5 – 23:20

Block 13

Block 0

DW5 – 19:16

Block 12

Block 0

DW5 – 15:12

Block 11

Block 0

DW5– 11:8

Block 10

Block 0

DW5 – 7:4

Block 9

Block 0

DW5 – 3:0

Block 8

Block 0

vctrl_pred_mode[63:0]

(vctrl_it_lumaintrapredmode3[15:0] & vctrl_it_lumaintrapredmode2[15:0] &

vctrl_it_lumaintrapredmode1[15:0] & vctrl_it_lumaintrapredmode0[15:0] ) :

vctrl_pred_mode_noextend[63:0]

(vctrl_INTRA_vld_16x16mode & vctrl_it_Transform8x8Flag) ?

vctrl_pred_mode_noextend_4x4[63:0] :

vctrl_pred_mode_noextend_16x16[63:0] :

vctrl_pred_mode_noextend_8x8[63:0] :

vctrl_pred_mode_noextend_4x4[63:0]

vctrl_pred_mode_noextend_16x16[63:0]

vctrl_it_lumaintrapredmode0[3:0] & vctrl_it_lumaintrapredmode0[3:0] &

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

vctrl_it_lumaintrapredmode0[3:0] & vctrl_it_lumaintrapredmode0[3:0] &

vctrl_it_lumaintrapredmode0[3:0] & vctrl_it_lumaintrapredmode0[3:0]

vctrl_pred_mode_noextend_8x8[63:0]

"h000" & vctrl_it_lumaintrapredmode0[15:12] &

"h000" & vctrl_it_lumaintrapredmode0[11:8] &

"h000" & vctrl_it_lumaintrapredmode0[7:4] &

"h000" & vctrl_it_lumaintrapredmode0[3:0]

vctrl_pred_mode_noextend_4x4[63:0]

vctrl_it_lumaintrapredmode3[15:0] & vctrl_it_lumaintrapredmode2[15:0] &

vctrl_it_lumaintrapredmode1[15:0] & vctrl_it_lumaintrapredmode0[15:0]

Inline data for RefPicSelect

ExtendedForm

0 or 1

16x16

16x8

8x16

8x8

16x16

16x8

8x16

DW8 – 31:24

L0 blk3

L0 blk0

L0 blk1

DW8 – 23:16

L0 blk2

L0 blk0

DW8 – 15:8

L0 blk1

L0 blk0

L0 blk1

DW8 – 7:0

L0 blk0

DW9 – 31:24

L1 blk3

L1 blk0

L1 blk1

DW9 – 23:16

L1 blk2

L1 blk0

DW9 – 15:8

L1 blk1

L1 blk0

L1 blk1

DW9 – 7:0

L1 blk0

The inline data content of Dwords 4 to 6 is defined either for intra prediction or for inter prediction, but

not both.

Inline data subfields for an Intra Macroblock

Dword

Bit

Description

31:16

LumaIntraMode[1]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block of a MB, 4-bit each.

See the bit assignment table later in this section.

15:0

LumaIntraMode[0]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block, four 8x8 block or one intra16x16 of

a MB.

4-bit per 4x4 sub-block (Transform8x8Flag=0, Mbtype=0 and intraMbFlag=1) or 8x8 block

(Transform8x8Flag=1, Mbtype=0, MbFlag=1), since there are 9 intra modes.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Dword

Bit

Description

4-bit for intra16x16 MB (Transform8x8Flag=0, Mbtype=1 to 24 and intraMbFlag=1), but only the

LSBit[1:0] is valid, since there are only 4 intra modes.

See the bit assignment table later in this section.

31:16

LumaIntraMode[3]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block of a MB, 4-bit each.

See the bit assignment table later in this section.

15:0

LumaIntraMode[2]

Specifies the Luma Intra Prediction mode for four 4x4 sub-block of a MB, 4-bit each.

See the bit assignment later in this section.

31:8

Reserved: MBZ

(Reserved for encocder turbo mode IntraResidueDataSize, when this is not 0, optional residue

data are provided to the PAK; Reserved for decoder)

7:0

IntraStruct

This field contains 6 bits for IntraPredAvailFlags[5:0] and 2 bits for ChromaIntraPredMode. The

IntraPredAvailFlags[4:0] (the lower 5 bits) have already included the effect of the

constrained_intra_pred_flag. See the diagram later for the definition of neighbor position around the

current MB or MB pair (in MBAFF mode).

1 – IntraPredAvailFlagY, indicates the values of samples of neighbor Y can be used in intra prediction

for the current MB.

0 – IntraPredAvailFlagY, indicates the values of samples of neighbor Y is not available for intra

prediction of the current MB.

IntraPredAvailFlag-A and -E can only be different from each other when constrained_intra_pred_flag

is equal to 1 and mb_field_decoding_flag is equal to 1 and the value of the mb_field_decoding_flag

for the macroblock pair to the left of the current macroblock is equal to 0 (which can only occur

when MbaffFrameFlag is equal to 1).

IntraPredAvailFlag-F is used only if

 It is in MBAFF mode, that is, MbaffFrameFlag = 1

 The current macroblock is of frame type, that is, MbFieldFag = 0

 The current macroblock type is Intra8x8, that is, IntraMbFlag = INTRA, IntraMbMode =

INTRA_8x8, and Transform8x8Flag = 1

In any other cases IntraPredAvailFlag-A shall be used instead.

Bits

IntraPredAvailFlags Definition

IntraPredAvailFlagF – F (Left 8

row (-1,7) neighbor)

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Dword

Bit

Description

IntraPredAvailFlagA – A (Left neighbor top half)

IntraPredAvailFlagE – E (Left neighbor bottom half)

IntraPredAvailFlagB – B (Top neighbor)

IntraPredAvailFlagC – C (Top right neighbor)

IntraPredAvailFlagD – D (Top left corner neighbor)

1:0

ChromaIntraPredMode – 2 bits to specify 1 of 4 chroma intra prediction modes, see the

table in later section.

Inline data subfields for an Inter Macroblock

DWord

Bit

Description

31:16

Reserved: MBZ

15:8

SubMbPredMode (Sub-Macroblock Prediction Mode): If InterMbMode is INTER8x8, this field

describes the prediction mode of the sub-partitions in the four 8x8 sub-macroblock. It contains four

subfields each with 2-bits, corresponding to the four 8x8 sub-macroblocks in sequential order.

This field is derived from sub_mb_type for a BP_8x8 macroblock.

This field is derived from MbType for a non-BP_8x8 inter macroblock, and carries redundant

information as MbType).

If InterMbMode is INTER16x16, INTER16x8 or INTER8x16, this field carries the prediction modes of

the sub macroblock (one 16x16, two 16x8 or two 8x16). The unused bits are set to zero.

Bits [1:0]: SubMbPredMode[0]

Bits [3:2]: SubMbPredMode[1]

Bits [5:4]: SubMbPredMode[2]

Bits [7:6]: SubMbPredMode[3]

7:0

SubMbShape (Sub Macroblock Shape)

This field describes the sub-block partitioning of each sub macroblocks (four 8x8 blocks). It contains

four subfields each with 2-bits, corresponding to the 4 fixed size 8x8 sub macroblocks in sequential

order.

This field is provided for MB with sub_mb_type equal to BP_8x8 only (B_8x8 and P_8x8 as defined in

DXVA). Otherwise, this field is ignored by hardware

Bits [1:0]: SubMbShape[0] – for 8x8 Block 0

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

Bits [3:2]: SubMbShape[1] – for 8x8 Block 1

Bits [5:4]: SubMbShape[2] – for 8x8 Block 2

Bits [7:6]: SubMbShape[3] – for 8x8 Block 3

Blocks of the MB is numbered as follows :

Each 2-bit value [1:0] is defined as :

00 – SubMbPartWidth=8, SubMbPartHeight=8

01 – SubMbPartWidth=8, SubMbPartHeight=4

10 – SubMbPartWidth=4, SubMbPartHeight=8

11 – SubMbPartWidth=4, SubMbPartHeight=4

31:24

RefPicSelect[0][3]

Support up to 4 reference pictures per L0 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List0 Table.

23:16

RefPicSelect[0][2]

Support up to 4 reference pictures per L0 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List0 Table.

15:8

RefPicSelect[0][1]

Support up to 4 reference pictures per L0 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List0 Table.

7:0

RefPicSelect[0][0]

Support up to 4 reference pictures per L0 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List0 Table.

31:24

RefPicSelect[1] [3]

Support up to 4 reference pictures per L1 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List1 Table.

For P- picture these bits must be set to zero.

23:16

RefPicSelect[1][2]

Support up to 4 reference pictures per L1 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List1 Table.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

For P- picture these bits must be set to zero.

15:8

RefPicSelect[1][1]

Support up to 4 reference pictures per L1 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List1 Table.

For P- picture these bits must be set to zero.

7:0

RefPicSelect[1][0]

Support up to 4 reference pictures per L1 direction, one per MB partition, if exists. See details in

later section. This field specifies the reference index into the Reference Picture List1 Table.

For P- picture these bits must be set to zero.

Luma Intra Prediction Modes

Luma Intra Prediction Modes (LumaIntraPredModes) is defined in Luma Intra Prediction Modes . It is

further categorized as Intra16x16PredMode (Luma Intra Prediction Modes ), Intra8x8PredMode (Luma

Intra Prediction Modes ) and Intra4x4PredMode (Luma Intra Prediction Modes ), operating on 16x16, 8x8

and 4x4 block sizes, respectively. illustrates the intra prediction directions geometrically for the Intra4x4

prediction. When a macroblock is subdivided, the intra prediction is performed for the subdivision in a

predetermined order. For example, Luma Intra Prediction Modes shows the block order for Intra4x4

prediction. And Luma Intra Prediction Modes shows the block order of Block8x8 in a 16x16 region or

Block4x4 in an 8x8 region.

Definition of LumaIntraPredModes

LumaIntraPredModes

[index]

Intra16x16PredMode

Intra8x8PredMode

Intra4x4PredMode

Index

Bit

MbType = [1…24]

Transform8x8Flag = 0

MbType = 0

Transform8x8Flag = 1

MbType = 0

Transform8x8Flag = 0

15:12

MBZ

Block8x8 3

Block4x4 3 (0_0)

11:8

MBZ

Block8x8 2

Block4x4 2 (0_1)

7:4

MBZ

Block8x8 1

Block4x4 1 (0_2)

3:0

Block16x16

Block8x8 0

Block4x4 0 (0_3)

15:12

MBZ

Block4x4 7 (1_0)

11:8

MBZ

Block4x4 6 (1_1)

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

LumaIntraPredModes

[index]

Intra16x16PredMode

Intra8x8PredMode

Intra4x4PredMode

Index

Bit

MbType = [1…24]

Transform8x8Flag = 0

MbType = 0

Transform8x8Flag = 1

MbType = 0

Transform8x8Flag = 0

7:4

MBZ

Block4x4 5 (1_2)

3:0

MBZ

Block4x4 4 (1_3)

15:12

MBZ

Block4x4 11 (2_0)

11:8

MBZ

Block4x4 10 (2_1)

7:4

MBZ

Block4x4 9 (2 2)

3:0

MBZ

Block4x4 8 (2_3)

15:12

MBZ

Block4x4 15 (3_0)

11:8

MBZ

Block4x4 14 (3_1)

7:4

MBZ

Block4x4 13 (3_2)

3:0

MBZ

Block4x4 12 (3_3)

Definition of Intra16x16PredMode

Intra16x16PredMode

Description

Intra_16x16_Vertical

Intra_16x16_Horizontal

Intra_16x16_DC

Intra_16x16_Plane

4 – 15

Reserved

Definition of Intra8x8PredMode

Intra8x8PredMode

Description

Intra_8x8_Vertical

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Intra8x8PredMode

Description

Intra_8x8_Horizontal

Intra_8x8_DC

Intra_8x8_Diagonal_Down_Left

Intra_8x8_Diagonal_Down_Right

Intra_8x8_Vertical_Right

Intra_8x8_Horizontal_Down

Intra_8x8_Vertical_Left

Intra_8x8_Horizontal_Up

9 – 15

Reserved

Definition of Intra4x4PredMode

Intra4x4PredMode

Description

Intra_4x4_Vertical

Intra_4x4_Horizontal

Intra_4x4_DC

Intra_4x4_Diagonal_Down_Left

Intra_4x4_Diagonal_Down_Right

Intra_4x4_Vertical_Right

Intra_4x4_Horizontal_Down

Intra_4x4_Vertical_Left

Intra_4x4_Horizontal_Up

9 – 15

Reserved

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Intra_4x4 prediction mode directions

Numbers of Block4x4 in a 16x16 region

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Numbers of Block4x4 in an 8x8 region or numbers of Block8x8 in a 16x16 region

Definition of Chroma Intra Prediction Mode

ChromaIntraPredMode

(intra_chroma_pred_mode)

Name of intra_chroma_pred_mode

Intra_Chroma_DC (prediction mode)

Intra_Chroma_Horizontal (prediction mode)

Intra_Chroma_Vertical (prediction mode)

Intra_Chroma_Plane (prediction mode)

Reference Indices Defined for Each MB Partition Type and Bit Assignment

frame/field MB/Picture

MB partitioning

16x16

16x8

8x16

8x8

RefIdxL0/1[0]

blk0

Bit 7:0

RefIdxL0/1[1]

blk1

Bit 15:8

RefIdxL0/1[2]

blk2

Bit 23:16

RefIdxL0/1[3]

blk3

Bit 31:24

MB Neighbor Availability in Intra-Prediction Modes (IntraPredAvailFlags)

Current MB is labelled as X. For non-MBAFF mode, 4 neighbors, A, B, C, D, are depicted in the following

picture and are defined as the following.

 MB D: top left neighbor of current MB X

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

 MB C: top right neighbor of current MB X

 MB B: top neighbor of current MB X

 MB A: left neighbor of the current MB X

mbAddrD

(top-left)

mbAddrB

(top)

mbAddrC

(top-right)

mbAddrA

(left)

CurrMbAddrX

N/A

For MBAFF mode, the current MB is labelled as X0 or X1, 4 neighbor pairs, A0/A1, B0/B1, C0/C1, D0/D1,

are depicted in the following picture and are defined as the following.

 MB D0: first MB of top left neighbor MB pair of current MB pair X0/X1

 MB D1: second MB of top left neighbor MB pair of current MB pair X0/X1

 MB C0: first MB of top right neighbor MB pair of current MB pair X0/X1

 MB C1: second MB of top right neighbor MB pair of current MB pair X0/X1

 MB B0: first MB of top neighbor MB pair of current MB pari X0/X1

 MB B1: second MB of top neighbor MB pair of current MB pari X0/X1

 MB A0: first MB of left neighbor MB pair of the current MB pair X0/X1

 MB A1: second MB of left neighbor MB pair of the current MB pair X0/X1

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

mbAddrD

mbAddrB

mbAddrC

mbAddrD+1

mbAddrB+1

mbAddrC+1

mbAddrA

CurrMbAddrX

N/A

mbAddrA+1

CurrMbAddrX

N/A

For a given macroblock X (or X0/X1), the 6 neighbor availability signals, namely, A, B, C, D, E, F, are

defined as the following.

 IntraPredAvailFlagF – F (Single neighbor pixel at the left 8th row (-1,7)

 IntraPredAvailFlagA – A (Left neighbor top half pixel group)

 IntraPredAvailFlagE – E (Left neighbor bottom half pixel group)

 IntraPredAvailFlagB – B (Top neighbor pixel group)

 IntraPredAvailFlagC – C (Top right neighbor pixel group)

 IntraPredAvailFlagD – D (Top left corner neighbor pixel)

The following table depicts the generation of IntraPredAvailFlags[5:0] signals in a condensed form. It

should note that for most cases only one input neighbor signal is assigned for each condition. The

exception is in the four places for deriving left neighbor A and E where the neighbor is only available if

left neighbors (A0 and A1) are both available (A0&A1). Also note that F takes output value very similar to

that for A except the two "AND" conditions, where F is assigned to A1 instead of (A0&A1).

Table: Definition of intra-prediction neighbor availability calculation in MBAFF mode

Output ⇒

Current X \

Neighbor Y

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

(Top)

X-Frame

X-Field

(Bottom)

X-Frame

N/A

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Output ⇒

Current X \

Neighbor Y

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

Y-

Frame

Y-

Field

X-Field

In MB Neighbor Availability in Intra-Prediction Modes (IntraPredAvailFlags) , X-Frame or X-Field indicates

the frame/field mode of the current MB; and Y-Frame or Y-Field indicates the corresponding neighbor

MB for the given neighbor location, being upper left (D) or left (A) for example. Therefore, "Y-" takes the

selected neighbor MB name as in the output cell in the same column. For example, for output D, if X1 is a

frame MB, Y = A, if X1 is a field MB, Y = D.

For non-MBAFF mode, as A0=A1, B0=B1, C0=C1 and D0=D1, the neighbor assignment is degenerated

into the following simple table. Here, E is assigned to the same as A and F is forced to 0.

Table: Definition of intra-prediction neighbor availability calculation in non-MBAFF mode

Output ⇒

To further explain the neighbor assignment rules in MB Neighbor Availability in Intra-Prediction Modes

(IntraPredAvailFlags) , the following table provides description for each condition. Please note that this

table is informative as it provides redundant information as in MB Neighbor Availability in Intra-

Prediction Modes (IntraPredAvailFlags) .

Table: Detailed explanation of intra-prediction neighbor availability calculation in MBAFF mode

Current

MB Field

Neighbor

MB Field

Neighbor

MB Select

(Y=?)

Neighbor Avail

Result

(OUTPUT)

Description

(Top)

X-Frame

Y-Frame

Top Frame MB uses [-1,-1] = D_31, thus D1

only, regardless D frame or field pair

X-Frame

Y-Field

X-Field

Y-Frame

Top Field MB uses [-1,-2] = D_30, thus if D is

frame pair, takes D1 (D1_14 pixel), and if D is

field pair, takes D0 (D0_15 pixel)

X-Field

Y-Field

(Bottom)

X-Frame

Y-Frame

Bottom Frame MB uses [-1,15] = A_15, thus

A0 (A0_15 pixel) if A is a frame pair, or A1

(A1_7 pixel), if A is a field pair

X-Frame

Y-Field

X-Field

Y-Frame

Bottom Field MB uses [-1,-1] = D_31, thus D1

only, regardless D frame or field pair

X-Field

Y-Field

(Top)

X-Frame

Y-Frame

Top Frame MB uses [0…15,-1] = B_31, thus

B1 only, regardless B frame or field pair

X-Frame

Y-Field

X-Field

Y-Frame

Top Field MB uses [0…15,-2] = B_30, thus if B

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Current

MB Field

Neighbor

MB Field

Neighbor

MB Select

(Y=?)

Neighbor Avail

Result

(OUTPUT)

Description

X-Field

Y-Field

is frame pair, takes B1 (B1_14 row), and if B is

field pair, takes B0 (B0_15 row)

(Bottom)

X-Frame

Y-Frame

Bottom Frame MB uses [0…15,15], thus X0

(X0_15 row)

X-Frame

Y-Field

n/a

Note: X0 and X1 must have the same field

type, this row is n/a.

X-Field

Y-Frame

Bottom Field MB uses [0…15,-1] = B_31, thus

B1 only, regardless B frame or field pair

X-Field

Y-Field

(Top)

X-Frame

Y-Frame

Top Frame MB uses [16…23,-1] = C_31, thus

C1 only, regardless C frame or field pair

X-Frame

Y-Field

X-Field

Y-Frame

Top Field MB uses [16…23,-2] = C_30, thus if

C is frame pair, takes C1 (C1_14 row), and if C

is field pair, takes C0 (C0_15 row)

X-Field

Y-Field

(Bottom)

X-Frame

Y-Frame

n/a

Bottom Frame MB doesn’t have left-top

neighbor by definition, thus forced to 0

X-Frame

Y-Field

n/a

X-Field

Y-Frame

Bottom Field MB uses [16…23,-1] = C_31,

thus C1 only, regardless C frame or field pair

X-Field

Y-Field

(Top)

X-Frame

Y-Frame

First Half of Top Frame MB uses [-1,0…7],

thus A0 if A is a frame pair; but is only avail if

both A0 and A1 are avail if A is a field pair

due to the mix

X-Frame

Y-Field

A0&A1

X-Field

Y-Frame

First Half of Top Field MB uses [-1,0..2..4..14],

thus take A0 (if A is frame pair, takes A0 even

lines, and if A is field pair, takes A0 first half)

X-Field

Y-Field

(Bottom)

X-Frame

Y-Frame

First Half of Bottom Frame MB uses [-

1,16…23], thus A1 if A is a frame pair; but is

only avail if both A0 and A1 are avail if A is a

field pair due to the mix

X-Frame

Y-Field

A0&A1

X-Field

Y-Frame

First Half of Bottom Field MB uses [-

1,1..3..15], thus take A0 (if A is frame pair,

takes A0 odd lines, and if A is field pair, takes

A1 first half)

X-Field

Y-Field

(Top)

X-Frame

Y-Frame

Second Half of Top Frame MB uses [-1,8…15],

thus A0 if A is a frame pair; but is only avail if

X-Frame

Y-Field

A0&A1

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Current

MB Field

Neighbor

MB Field

Neighbor

MB Select

(Y=?)

Neighbor Avail

Result

(OUTPUT)

Description

both A0 and A1 are avail if A is a field pair

due to the mix

X-Field

Y-Frame

Second Half of Top Field MB uses [-

1,16..18..30], thus take A1 (if A is frame pair,

takes A1 even lines, and if A is field pair,

takes A0 second half)

X-Field

Y-Field

(Bottom)

X-Frame

Y-Frame

Second Half of Bottom Frame MB uses [-

1,24…31], thus A1 if A is a frame pair; but is

only avail if both A0 and A1 are avail if A is a

field pair due to the mix

X-Frame

Y-Field

A0&A1

X-Field

Y-Frame

Second Half of Bottom Field MB uses [-

1,17..19..31], thus takes A1 (if A is frame pair,

takes A1 odd lines, and if A is field pair, takes

A1 second half)

X-Field

Y-Field

(Top)

X-Frame

Y-Frame

Top Frame MB uses [-1,7] = A_7 (odd

location), thus A0 if A is frame pair and A1 if

field pair

X-Frame

Y-Field

X-Field

Y-Frame

Top Field MB uses [-1,14] = A_14 (even

location), thus A0 regardless A frame or field

pair

X-Field

Y-Field

(Bottom)

X-Frame

Y-Frame

Bottom Frame MB uses [-1,23] = A_23 (odd

location), thus A1 regardless A frame or field

pair

X-Frame

Y-Field

X-Field

Y-Frame

Bottom Field MB uses [-1,15] = A_15 (odd

location), thus A0 if A is frame pair and A1 if

A is field pair

X-Field

Y-Field

Macroblock Type for Intra Cases

MbType follows two different tables according to whether the macroblock is an inter or intra

macroblock according to IntraMbFlag.

For an intra macroblock, MbType, as defined in Macroblock Type for Intra Cases , carries redundant

information as IntraMbMode. The notation I_16x16_x_y_z used in the table, ‘x’ is

Intra16x16LumaPredMode, ‘y’ is ChromaCbpInd, and ‘z’ is LumaCbpInd, as defined in Macroblock Type for

Intra Cases .

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MbType definition for Intra Macroblock

Macroblock Type

MbType

I_4x4

I_8x8

I_16x16_0_0_0

I_16x16_1_0_0

I_16x16_2_0_0

I_16x16_3_0_0

I_16x16_0_1_0

I_16x16_1_1_0

I_16x16_2_1_0

I_16x16_3_1_0

I_16x16_0_2_0

I_16x16_1_2_0

I_16x16_2_2_0

I_16x16_3_2_0

I_16x16_0_0_1

I_16x16_1_0_1

I_16x16_2_0_1

I_16x16_3_0_1

10h

I_16x16_0_1_1

11h

I_16x16_1_1_1

12h

I_16x16_2_1_1

13h

I_16x16_3_1_1

14h

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Macroblock Type

MbType

I_16x16_0_2_1

15h

I_16x16_1_2_1

16h

I_16x16_2_2_1

17h

I_16x16_3_2_1

18h

I_PCM

19h (used by HW)

Note: MbType here is identical as specified in DXVA 2.0.

For Intra_16x16 modes, the 5 bits of value (MbType – 1) have the following meanings.

Sub field definition used by MbType for a macroblock with Intra16x16 prediction

Bits

Description

LumaCbpInd – Luma Coded Block Pattern Indicator

0 means none of the luma blocks are coded. 1 means that at least one luma

block is coded.

0 = SUBMODE_I16_L_0

1 = SUBMODE_I16_L_NZ

In VME output, this field is forced to be 1 before adding 1 in Intra_16x16 mode.

3:2

ChromaCbpInd – Chroma Coded Block Pattern Indicator

00 means none of chroma blocks are coded. 01 means that only the chroma DC

block is coded, but all AC blocks are not coded. 10 means that at least one AC

chroma block is coded.

00 = SUBMODE_I16_C_0

01 = SUBMODE_I16_C_DC

10 = SUBMODE_I16_C_NZ

11 = Reserved

In VME output, this field is forced to be 10 before adding 1 in Intra_16x16 mode.

Programming Note: Adding 1 to MbType by VME hardware may have carry in to

this field. But as ‘11’ is reserved, the carry-in doesn’t propagate into bit 4 or higher.

This allows software to update MbType, if desired, using the redundant

LumaIntraPredModes information.

1:0

Intra16x16PredMode – Intra16x16 Prediction Mode

These two bits carries redundant (identical) information as that in

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Bits

Description

LumaIntraPredModes[0][0].

0 = SUBMODE_I16_VER

1 = SUBMODE_I16_HOR

2 = SUBMODE_I16_DC

3 = SUBMODE_I16_PLANE

IntraMbMode definition

IntraMbMode [1:0]

Description

Supported by VME?

Used by PAK?

INTRA_16x16 (redundant with MbType)

Yes

Ignored

INTRA_8x8

Yes

INTRA_4x4

Yes

IPCM (redundant with MbType)

Ignored

As an alternative representation, MbType is logically the same as the following, except the I_PCM and

I_NxN (i.e. I_4x4 and I_8x8) cases:

 24 types of 16x16 intra modes: A+B+C+D: (1h – 18h)

MBTYPE_INTRA_16x16

o 4 Intra16x16 modes:

SUBMODE_I16_VER

SUBMODE_I16_HOR

SUBMODE_I16_DC

SUBMODE_I16_PLN

o 3 Chroma Cbp indices:

SUBMODE_I16_C_0

SUBMODE_I16_C_DC

SUBMODE_I16_C_NZ

o 2 Luma Cbp indices:

SUBMODE_I16_L_0

SUBMODE_I16_L_NZ

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Macroblock Type for Inter Cases

Sub-Macroblock Prediction Mode, SubMbPredMode, indicates the prediction mode for the sub-

partitions. Prediction mode specifies prediction direction being forward (from L0), backward (from L1) or

bi-directional (from both L0 and L1). Its meaning depends on InterMbMode. Macroblock Type for Inter

Cases provides the definition of the field.

 If InterMbMode is INTER16x16, only SubMbPredMode[0] is valid, it describes the prediction mode

of the 16x16 macroblock. The other entries are set to zero by hardware.

o For AVC, SubMbPredMode[0] contains redundant information as encoded in MbType

parameter.

o Note: SubMbPredMode[1]-[3] are intentionally set to zero to allow a simple LUT to derive

MbType as described later.

 If InterMbMode is INTER16x8, and INTER8x16, only the first two entries SubMbPredMode[0] and

SubMbPredMode[1] are valid, describing the sub-macroblock prediction mode.

o For AVC, SubMbPredMode[0]/[1] contains redundant information as encoded in MbType

parameter.

o Note: SubMbPredMode[2]-[3] are intentionally set to zero to allow a simple LUT to derive

MbType as described later.

 If InterMbMode is INTER8x8, each entry of SubMbPredMode describes the prediction mode of the

sub-partition of an 8x8 sub-macroblock.

o For AVC, SubMbPredMode can be derived from sub_mb_type field for BP_8x8

macroblocks as defined in AVC spec.

o Note on Direct Sub-macroblock Prediction Mode: Direct prediction is not conveyed through

SubMbPredMode, instead, it is carried through Direct8x8Pattern.

InterMbMode definition

MbSkipFlag

InterMbMode

Description

INTER16x16

INTER16x8

INTER8x16

INTER8x8

PSKIP/BSKIP16x16*

BSKIP

1, 2

Reserved

Used by PAK

Ignored by PAK

* BSKIP16x16 is an optional non-standard but equivalent optimization.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Definition of SubMbPredMode based on InterMbMode

SubMbPredMode

INTER16x16

INTER16x8

INTER8x16

INTER8x8

Bit

MbType = [1…3]

MbType = [16h]

MbType = [4…15h]

MbType = [16h]

7:6

MBZ

Block8x8 3

5:4

MBZ

Block8x8 2

3:2

MBZ

Block16x8 1

Block8x16 1

Block8x8 1

1:0

Block16x16

Block16x8 0

Block8x16 0

Block8x8 0

Ignored by PAK

Used by PAK

Definition of SubMbPredMode[i]

SubMbPredMode

Description

InterMbMode

VME Output

MvCountPred

Notes

Pred_L0

All

Yes

P or B Slice

Pred_L1

All

Yes

B Slice Only

BiPred

All

Yes

B Slice Only

Reserved

Sub-Macroblock Shape, SubMbShape[i], for i = 0…3, describes the shape of the sub partitions of the 8x8

sub-macroblock of a BP_8x8 macroblock. This field is only valid if InterMBMode is INTER8x8. They are

defined in Macroblock Type for Inter Cases . The parameters can be derived from sub_mb_type field as

defined in AVC spec.

Note: These fields must be correctly set even for Direct or Skip 8x8 cases, the individual B_Direct_8x8

block is flagged by the Direct8x8Pattern variable.

Definition of SubMbShape for an 8x8 region of a BP_8x8 macroblock (including BSKIP, BDIRECT)

Description

SubMbShape

NumSubMbPart

SubMbPartWidth

SubMbPartHeight

MvCountShape

For an inter macroblock, MbType, carries redundant information as InterMbMode and SubMbPredMode.

Macroblock Type for Inter Cases provides the typical inter macroblock types and Macroblock Type for

Inter Cases provides that with skip and direct modes. The definition of MbType for both P slice and B

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

slice is the same and is equivalent to that for mb_type of a B slice in the AVC spec. As direct mode is

indicated using a separate field Direct8x8Pattern, 0 is reserved for MbType.

Here, MVCount is the number of motion vectors actually encoded in the bitstream. It is informative. For a

BP_8x8 or equivalent Skip/Direct macroblock, MVCount is the sum of the following term for the four 8x8

sub macroblock (with i = 0…3):

MvCountShape[i] * MvCountPred[i] * MvCountDirect[i]

where MvCountShape[i] is block count for sub macroblock [i], MvCountPred[i] is the motion vector count

for each block of sub macroblock[i], and MvCountDirect[i] is the multipler for direct mode for B Slice,

indicating whether motion vectors are coded or not. It must be set to 1 for P slice. For B Slice,

MvCountDirect[i] = !Direct8x8Pattern[i], which is 0 for a sub macroblock coded as direct mode and 1

otherwise.

In the tables, "DC" stands for "Don’t Care" as PAK hardware ignores these fields.

MbType definition for Inter Macroblock (and MbSkipflag = 0)

Macroblock Type

MbType

MbSkipFlag

Direct8x8Pattern

SubMbShape

SubMbPredMode

MVCount

Reserved

BP_L0_16x16

B_L1_16x16

B_Bi_16x16

BP_L0_L0_16x8

BP_L0_L0_8x16

B_L1_L1_16x8

B_L1_L1_8x16

B_L0_L1_16x8

B_L0_L1_8x16

B_L1_L0_16x8

0Ah

B_L1_L0_8x16

0Bh

B_L0_Bi_16x8

0Ch

B_L0_Bi_8x16

0Dh

B_L1_Bi_16x8

0Eh

B_L1_Bi_8x16

0Fh

B_Bi_L0_16x8

10h

B_Bi_L0_8x16

11h

B_Bi_L1_16x8

12h

B_Bi_L1_8x16

13h

B_Bi_Bi_16x8

14h

B_Bi_Bi_8x16

15h

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Macroblock Type

MbType

MbSkipFlag

Direct8x8Pattern

SubMbShape

SubMbPredMode

MVCount

BP_8x8

16h

!= Fh

vary

Sum

Reserved

17h-1Fh

Additional MbType definition with Direct/Skip for Inter Macroblock

Macroblock Type

Type

Xfrm

8x8

MbSkip

Flag

Direct8x8

Pattern

SubMb

Shape

SubMb

PredMode

MvCount

Notes

P_Skip_16x16

Skipped

macroblock.

Motion

compensation like

P_L0_16x16

B_Skip_16x16_4MVPair

16h

vary

Skipped

macroblock.

Motion

compensation like

B_8x8 with 8x8

subblocks, when

direct_8x8_infere

nce_flag is set to 1

B_Skip_16x16_16MVPair

16h

FFh

vary

Skipped

macroblock.

Motion

compensation like

B_8x8 with 4x4

subblocks, when

direct_8x8_infere

nce_flag is set to 0

B_Direct_16x16_4MVPai

16h

vary

MbType coded as

B_Direct_16x16.

Motion

compensation like

B_8x8 with 8x8

subblocks, when

direct_8x8_infere

nce_flag is set to 1

B_Direct_16x16_16MVP

air

16h

FFh

vary

MbType coded as

B_Direct_16x16.

Motion

compensation like

B_8x8 with 4x4

subblocks, when

direct_8x8_infere

nce_flag is set to 0

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

People might notice that B_DIRECT_16x16 and B_SKIP are mapped on BP_8x8 for AVC decoding interface

in IT mode as the motion compensation operation for both modes are the same as BP_8x8. According to

AVC Spec, motion vectors for B_DIRECT_16x16 and B_SKIP are derived from temporally co-located

macroblock on an 8x8 sub macroblock basis if direct_8x8_inference_flag is set to 1 or on a 4x4 block

basis if it is set to 0. For each sub macroblock or block, SubMbPredMode is derived, thus can any of the

valid numbers. Motion vectors may also be different. In spatial direct mode, the motion vectors are

subject to spatial neighbor macroblocks as well as co-located macroblock. The spatial prediction is based

on the neighbor macroblocks, so the same spatial predicted motion vector applies to all sub

macroblocks or blocks. However, under certain conditions, temporal predictor may replace (colZeroFlag)

the spatial predictor for a given sub macroblock or block. Thus the motion vectors may differ.

In Macroblock Type for Inter Cases , the macroblock type names for major partitions nicely follow forms

BP_MbPredMode_MbShape (like BP_L0_16x16) and B_MbPredMode0_MbPredMode1_MbShape (like

B_L0_Bi_16x8). For minor partitions it is fixed as BP_MbShape as BP_8x8.

However, in Macroblock Type for Inter Cases the macroblock types for Skip and Direct modes does not

follow the same rule. The third field in P_Skip_16x16 or B_Direct_16x16_x indicates that "Skip" or "Direct"

applies to the entire 16x16 macroblock, even though MbShape is 8x8 as that in BP_8x8. In order to

distinguish the SubMbShape being 8x8 or 4x4 for B_Skip and B_Direct, the fourth field is added. 4MVPair

indicates upto 4 MV pairs are presented with SubMbShape equals to 0; and 16MVPair indicates up to 16

MV pairs are presented with SubMbShape equals to FFh.Also note that P_8x8ref0 is not specified in PAK

input interface, it is up to hardware to detect and choose its packing format based on number of

reference indices and reference index for the given macroblock.

Macroblock Type Conversion Rules

For improved coding efficiency the PAK hardware has the capability to convert macroblock types to use

more efficiency coding modes such as DIRECT and SKIP. For an inter macroblock or a sub macroblock

coded as DIRECT, no motion vector is needed in the bitstream for the macroblock or sub macroblock. If a

macroblock is coded as SKIP, it only consumes one SKIP bit (no motion vector, no coefficients are coded).

And infomaton about the macroblock is ‘inferred’ according to the rules stated in the AVC Spec.

As the input to PAK, the following signals can convey the information regarding DIRECT and SKIP:

 MbSkipFlag

 Direct8x8Pattern

 CodecBlockPattern (CbpY, CbpCb, CbpCr)

Such conversion can be enabled or disabled through the SLICE_STATE fields DirectConvDisable and

SkipConvDisable as well as the in line command field MbSkipConvDisable.

A P slice doesn’t support direct mode, it only supports P_Skip, which is equivalent to a 16_16_L0

prediction. Other prediction types cannot be converted to P_Skip. The following table describes the

macroblock type conversion rules for a P slice. Here CBP = CbpY/CbpCb/CbpCr are the final computed

results after quantization by the hardware. Note that hardware honors the input CbpY/CbpCb/CbpCr

fields - if the value corresponding to a block is set to zero, the resulting CBP is also zero. The output

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

mb_skip_flag and mb_type are the symbols coded in the bitstream as defined in the AVC spec. DC stands

for Don't care, T for True.

Note that the internal condition of MV==MVP is subject to the precise rules stated in the AVC Spec as

quoted below. Note that there are exceptions for P_Skip from the normal motion vector prediction rules.

Derivation process for luma motion vectors for skipped macroblocks in P and SP slices

This process is invoked when mb_type is equal to P_Skip.

Outputs of this process are the motion vector mvL0 and the reference index refIdxL0.

The reference index refIdxL0 for a skipped macroblock is derived as follows.

refIdxL0 = 0. (8-168)

For the derivation of the motion vector mvL0 of a P_Skip macroblock type, the following applies.

- The process specified in subclause 8.4.1.3.2 is invoked with mbPartIdx set equal to 0, subMbPartIdx set

equal to 0, currSubMbType set equal to "na", and listSuffixFlag set equal to 0 as input and the output is

assigned to mbAddrA, mbAddrB, mvL0A, mvL0B, refIdxL0A, and refIdxL0B.

- The variable mvL0 is specified as follows.

- If any of the following conditions are true, both components of the motion vector mvL0 are set equal

to 0.

- mbAddrA is not available

- mbAddrB is not available

- refIdxL0A is equal to 0 and both components of mvL0A are equal to 0

- refIdxL0B is equal to 0 and both components of mvL0B are equal to 0

- Otherwise, the derivation process for luma motion vector prediction as specified in subclause 8.4.1.3 is

invoked with mbPartIdx = 0, subMbPartIdx = 0, refIdxL0, and currSubMbType = "na" as inputs and the

output is assigned to mvL0.

NOTE - The output is directly assigned to mvL0, since the predictor is equal to the actual motion vector.

Macroblock type conversion rule for an inter macroblock in a P slice

Input

Internal

Output

Notes

Macroblock

Type

SkipConvDisable

CBP

MVP

MbAffSkipAllowed

mb_skip_flag

mb_type

P_Skip_16x16

Forced to P_Skip;

Hardware will force

CBP to zero and also

ignore

SkipConvDisable

control. Hardware

doesn't check for

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Input

Internal

Output

Notes

Macroblock

Type

SkipConvDisable

CBP

MVP

MbAffSkipAllowed

mb_skip_flag

mb_type

MV==MVP error

condition

P_Skip_16x16

Reverse convert to

P_L0_16x16;

Hardware will force

CBP to zero but

reversely convert

MbType as

P_L0_16x16 once it

determines that Skip

is not allowed.

BP _16x16_L0

Converted to P_Skip.

Even input doesn't

provide skip hint,

hardware can

performance the

optimization by

detecting CBP and

MV==MVP

condition.

BP _16x16_L0

Reverse back to

P_L0_16x16;

Hardware will

reverse back to

P_L0_16x16 even

Skip conditions are

met once it

determines that Skip

is not allowed.

BP _16x16_L0

Still coded as

P_L0_16x16 = 0.

A B slice supports both direct and skip modes. The following table describes the macroblock type

conversion rules for a B slice. Hardware does not verify MV==MVP condition for a Skip/Direct

macroblock in a B Slice as no DMV is performed by hardware.

Macroblock type conversion rule for an inter macroblock in a B slice

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Input

Internal

Output

Notes

Macroblock Type

SkipConvDi

sable ||

SkipConvDi

sable

DirectConvDi

sable

MbAffSkipAll

owed

mb_skip_

flag

mb_ty

B_Skip_8x8

B_Skip_4x4

n/a

Forced to

B_Skip;

Hardware

will force

CBP to zero

and also

ignore

SkipConvDi

sable

control.

B_Skip_8x8

B_Skip_4x4

n/a

REVERSE

convert to

B_Direct_16

x16;

Hardware

will force

CBP to zero

and also

reverse

convert to

B_Direct_16

x16 when it

discovers

Skip is not

allowed.

B_Direct_16x16_4MVPair/

16MVPair

n/a

Converted

to B_Skip.

Hardware

first

converts to

B_Direct_16

x16 and

then further

to B_Skip if

CBP = 0.

B_Direct_16x16_4MVPair/

16MVPair

n/a

Converted

B_Direct_16

x16.

Hardware

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Input

Internal

Output

Notes

Macroblock Type

SkipConvDi

sable ||

SkipConvDi

sable

DirectConvDi

sable

MbAffSkipAll

owed

mb_skip_

flag

mb_ty

first

converts to

B_Direct_16

x16 and

stop there

as it

discovers

Skip is not

allowed

even

CBP=0.

B_Direct_16x16_4MVPair/

16MVPair

n/a

Converted

B_Direct_16

x16.

Hardware

converts to

B_Direct_16

x16 and

stops there

even

though CBP

= 0 as input

disallows

Skip

conversion.

B_Direct_16x16_4MVPair/

16MVPair

n/a

Converted

B_Direct_16

x16.

Hardware

converts to

B_Direct_16

x16 and

stops there

because

CBP != 0.

B_Direct_16x16_4MVPair/

16MVPair

n/a

16h

Stay as

B_8x8.

Hardware

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Input

Internal

Output

Notes

Macroblock Type

SkipConvDi

sable ||

SkipConvDi

sable

DirectConvDi

sable

MbAffSkipAll

owed

mb_skip_

flag

mb_ty

stays at

B_8x8 and

codes each

sub

macroblock

s even all

are direct.

The internal signal MbAffSkipAllowed is added to deal with a restriction on the frame/field flag

(MbFieldFlag) which is unique to MBAFF. MbAffSkipAllowed is always set to 1 in non-MBAFF modes. In

MBAFF mode, a macroblock pair may be both skipped only if its MbFieldFlag is the same as its available

neighbor macroblock pair A or B if A or B is available (in that order), or is not 0 if A/B are both not

available. Otherwise, one of the macroblocks in the pair must be coded.

To reduce the burden on software, PAK hardware handles the above restriction correctly. For the first MB

in a pair, MbAffSkipAllowed is always set to 1. Therefore, hardware allows converting the first MB to Skip

if skip conversion is enabled. For the second MB in a pair, hardware sets MbAffSkipAllowed to 0 if the

following is true:

 The current MB Pair has different MbFieldFlag than its available neighbor A or B if A or B is

available, or is not 0 if A/B are both not available

 And the first MB is coded as a SKIP (could be forced or converted)

Otherwise, it sets MbAffSkipAllowed to 1. As MbAffSkipAllowed is to 0 for the above condition, hardware

will disallow Skip mode for the second MB. If the input signal forces it to Skip, hardware performs

reverse-convertion to code it as P_L0_16x16 or B_Direct_16x16 with CBP = 0 for a macroblock in a P or B

Slice. This means that hardware is able to correct the programming mistake by software. If the

macroblock is not forced to skip, hardware simply disallows Skip conversion.

Software still has an option to disallow Skip Conversion on a per-MB basis using the MbSkipConvDisable

control field in the inline command.

Indirect Data Description

For each macroblock, an ENC-PAK data set consists of two types of data blocks: indirect MV data block

and inline MB information.

The indirect MV data block may be in two modes: unpackedmode and packed-size mode.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Unpacked Motion Vector Data Block

In the unpacked mode, motion vectors are expanded (or duplicated) to either bidirectional 8x8 8MV

major partition format, or bidirectional 4x4 32MV format. Thus either 32 bytes or 128 bytes is assigned to

each MB.

Motion Vector block contains motion vectors in an intermediate format that is partially expanded

according to the sub- macroblock size. During the expansion, a place that does not contain a motion

vector is filled by replicating the relevant motion vector according to the following motion vector

replication rules. If the relevant motion vector doesn’t exist (for the given L0 or L1), it is zero filled.

Motion Vector Replication Rules:

 Rule #1

o #1.1: For L0 MV, for any sub-macroblock or sub-partition where there is at least one

motion vector

 If L0 inter prediction exists, the corresponding L0 MV is used

 Else it must be zero

o #1.2: For L1 MV, for any sub-macroblock or sub-partition where there is at least one

motion vector

 If L1 inter prediction exists, the corresponding L1 MV is used

 Else it must be zero

 For a macroblock with a 16x16, 16x8 or 8x16 sub-macroblock, MvSize = 8. The eight MV fields

follow Rule #1.

o The 16x16 is broken down into 4 8x8 sub-macroblocks. The 16x16 MVs (after rule #1) are

replicated into all 8x8 blocks.

o For an 8x16 partition, each 8x16 is broken down into 2 8x8 stacking vertically. The 8x16

MVs (after rule #1) are replicated into both 8x8 blocks.

o For a 16x8 partition, each 16x8 is broken down into 2 8x8 stacking horizontally. The 16x8

MVs (after rule #1) are replicated into both 8x8 blocks.

 For macroblock with sub-macroblock of 8x8 without minor partition (SubMbShape[0…3] = 0),

MvSize = 8, (e.g. mb_type equal to P_8x8, P_8x8ref0, or B_8x8)

o There is no motion vector replication

 For macroblock with sub-macroblock of 8x8 with at least one minor partition (if any

SubMbShape[i] != 0), MvSize = 32, (e.g. mb_type equal to P_8x8, P_8x8ref0, or B_8x8)

o For an 8x8 sub-partition, the 8x8 MVs (after rule #1) is replicated into all the four 4x4

blocks.

o For an 4x8 sub-partition within an 8x8 partition, each 4x8 is broken down into 2 4x4

stacking vertically. The 4x8 MVs (after rule #1) are replicated into both 4x4 blocks.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

o For an 8x4 sub-partition within an 8x8 partition, each 8x4 is broken down into 2 4x4

stacking horizontally. The 8x4 MVs (after rule #1) are replicated into both 4x4 blocks.

o For a 4x4 sub-partition within an 8x8 partition, each 4x4 has its own MVs (after rule #1).

Motion Vector block and MvSize

DWord

Bit

MvSize

W1.0

31:16

MV_Y0_L0.y

MV_Y0_0_L0.y

15:0

MV_Y0_L0.x

MV_Y0_0_L0.x

W1.1

31:16

MV_Y0_L1.y

MV_Y0_0_L1.y

15:0

MV_Y0_L1.x

MV_Y0_0_L1.x

W1.2

31:0

MV_Y1_L0

MV_Y0_1_L0

W1.3

31:0

MV_Y1_L1

MV_Y0_1_L1

W1.4

31:0

MV_Y2_L0

MV_Y0_2_L1

W1.5

31:0

MV_Y2_L1

MV_Y0_2_L0

W1.6

31:0

MV_Y3_L0

MV_Y0_3_L0

W1.7

31:0

MV_Y3_L1

MV_Y0_3_L1

W2.0

31:0

n/a

MV_Y1_0_L1

W2.1

31:0

n/a

MV_Y1_0_L0

W2.2

31:0

n/a

MV_Y1_1_L1

W2.3

31:0

n/a

MV_Y1_1_L0

W2.4

31:0

n/a

MV_Y1_2_L1

W2.5

31:0

n/a

MV_Y1_2_L0

W2.6

31:0

n/a

MV_Y1_3_L0

W2.7

31:0

n/a

MV_Y1_3_L1

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

MvSize

W3.0

31:0

n/a

MV_Y2_0_L1

W3.1

31:0

n/a

MV_Y2_0_L0

W3.2

31:0

n/a

MV_Y2_1_L1

W3.3

31:0

n/a

MV_Y2_1_L0

W3.4

31:0

n/a

MV_Y2_2_L1

W3.5

31:0

n/a

MV_Y2_2_L0

W3.6

31:0

n/a

MV_Y2_3_L0

W3.7

31:0

n/a

MV_Y2_3_L1

W4.0

31:0

n/a

MV_Y3_0_L1

W4.1

31:0

n/a

MV_Y3_0_L0

W4.2

31:0

n/a

MV_Y3_1_L1

W4.3

31:0

n/a

MV_Y3_1_L0

W4.4

31:0

n/a

MV_Y3_2_L1

W4.5

31:0

n/a

MV_Y3_2_L0

W4.6

31:0

n/a

MV_Y3_3_L0

W4.7

31:0

n/a

MV_Y3_3_L1

The motion vector(s) for a given sub-macroblock or a sub-partition are uniquely placed in the output

message as shown by the non-duplicate fields in Unpacked Motion Vector Data Block and Unpacked

Motion Vector Data Block.

MV_Yx_L0 and MV_Yx_L1 may be present individually or both. If one is not present, the corresponding

field must be zero. Subsequently, the duplicated fields will be zero as well.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Motion Vector duplication by sub-macroblocks for a 16x16 macroblock, whereas the 8x8 column

is for 4x(8x8) partition without minor shape

DWord

Bit

16x16

16x8

8x16

8x8

W1.0

31:16

MV_Y0_L1 (A)

MV_Y0_L1

15:0

MV_Y0_L0 (A)

MV_Y0_L0

W1.1

31:16

Duplicate (A)

MV_Y1_L1

15:0

Duplicate (A)

MV_Y1_L0

W1.2

31:16

Duplicate (A)

MV_Y2_L1 (B)

Duplicate (A)

MV_Y2_L1

15:0

Duplicate (A)

MV_Y2_L0 (B)

Duplicate (A)

MV_Y2_L0

W1.3

31:16

Duplicate (A)

Duplicate (B)

MV_Y3_L1

15:0

Duplicate (A)

Duplicate (B)

MV_Y3_L0

Motion Vector duplication by sub-partitions for the first 8x8 sub-macroblock Y0 if any Y0-Y3

contains minor shape (Y1_ to Y3_ have the same format in W2 to W4)

DWord

Bit

8x8

8x4

4x8

4x4

W1.0

31:16

MV_Y0_L1

MV_Y0_0_L1 (A)

MV_Y0_0_L1

15:0

MV_Y0_L0

MV_Y0_0_L0 (A)

MV_Y0_0_L0

W1.1

31:16

Duplicate (A)

MV_Y0_1_L1 (B)

MV_Y0_1_L1

15:0

Duplicate (A)

MV_Y0_1_L0 (B)

MV_Y0_1_L0

W1.2

31:16

Duplicate (A)

MV_Y0_2_L1 (B)

Duplicate (A)

MV_Y0_2_L1

15:0

Duplicate (A)

MV_Y0_2_L0 (B)

Duplicate (A)

MV_Y0_2_L0

W1.3

31:16

Duplicate (A)

Duplicate (B)

MV_Y0_3_L0

15:0

Duplicate (A)

Duplicate (B)

MV_Y0_3_L1

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Packed-Size Motion Vector Data Block

In the packed case, no redundant motion vectors are sent. So the number of motion vectors sent, as

specified by MvQuantity is the same as the motion vectors that will be packed (MvPacked).

The following tables are for information only. Fields like MvQuantity and MvPacked are not required

interface fields.

MbSkipFlag

MbType

Description

Quantity

MvSize

(Minimal MvSize)

P_Skip_16x16

BP_L0_16x16

B_L1_16x16

B_Bi_16x16

BP_L0_L0_16x8

BP_L0_L0_8x16

B_L1_L1_16x8

B_L1_L1_8x16

B_L0_L1_16x8

B_L0_L1_8x16

0Ah

B_L1_L0_16x8

0Bh

B_L1_L0_8x16

0Ch

B_L0_Bi_16x8

0Dh

B_L0_Bi_8x16

0Eh

B_L1_Bi_16x8

0Fh

B_L1_Bi_8x16

10h

B_Bi_L0_16x8

11h

B_Bi_L0_8x16

12h

B_Bi_L1_16x8

13h

B_Bi_L1_8x16

14h

B_Bi_Bi_16x8

15h

B_Bi_Bi_8x16

16h

BP_8x8

³4

8 or 32

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

When MbType = 22, BP_8x8, take the sum of four individual 8x8 subblocks

Direct8x8Pattern

SubMb

Shape

SubMb

PredMode

Description

Quantity

Size

(Min MvSize)

ADD

P_Skip_8x8

B_Direct_L0_8x8

(B-Skip_ L0_8x8)

B_Direct_L1_8x8

(B-Skip_ L1_8x8)

B_Direct_Bi_8x8

(B-Skip_ Bi_8x8)

P_Skip_4x4

B_Direct_L0_4x4

(B-Skip_ L0_4x4)

B_Direct_L1_4x4

(B-Skip_ L1_4x4)

B_Direct_Bi_4x4

(B-Skip_ Bi_4x4)

BP_L0_8x8

B_L1_8x8

B_BI_8x8

BP_L0_8x4

B_L1_8x4

B_BI_8x4

BP_L0_4x8

B_L1_4x8

B_BI_4x8

BP_L0_4x4

B_L1_4x4

B_BI_4x4

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Macroblock Level Rate Control

The QRC (Quantization Rate Control) unit receives data from BSP (Bit Serial Packer) and VIN (Video In)

and generates adjustments to QP values across macroblocks.

QRC can be logically partitioned into two units as shown below.

Macroblock level rate control is handled by the RC logic and the quantization logic.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

The signals QPmod and panic are generated by the RC logic based on data feedback from BSP. A

flowchart of the RC logic is given below.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Theory of Operation Overview

BSP will generate a byte estimate for each macroblock packed. Additionally, the user will specify a target

and max size per macroblock. The running sum of these signals (actual, target, max) creates “curves”

which are used to identify when QP adjustments are necessary (see figure below). Three more curves are

symmetrically generated by QRC (upper_midpt, lower_midpt, sum_min) from target and max. The values

of target and max are specified by the user will dictate the shape of these curves.

The difference between sum_actual and sum_target (called ‘bytediff’) identifies the margin of error

between the target and actual sizes. The difference between the current bytediff and the previously

calculated bytediff represents the rate of change in this margin over time. The sign of this rate is used to

identify if the correction is trending in the appropriate direction (towards bytediff = 0).

QPmod

Each macroblock will have a requested QP (which could vary across macroblocks or remain constant).

QPmod is to be added to the QP requested. QPmod will be positive when the target was under-

predicted and negative when the target is over-predicted.

QPmod is incremented or decremented when internal counters (called ‘over’ and ‘under’) reach tripping

points (called ‘grow’ and ‘shrink’). For each MB processed and based on which region (1-6) sum_actual

falls in, various amounts of points are added to either counters. If over exceeds grow, QPmod is

incremented whereas if under exceeds shrink, QPmod is decremented.

To dampen the effect of repeated changes in the same direction, an increase in resistance for that

direction and decrease in resistance for the complementary direction occurs (called ‘grow_resistance’ and

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

‘shrink_resistance’). This resistance is added to grow or shrink, which then requires more points to trip

the next correction in that direction.

The user can specify guard-bands that limit the amount QPmod can be modified. QPmod cannot exceed

QPmax_pos_mod or become less than -QPmax_neg_mod_abs.

Triggering

The RC unit begins to modify QPmod occurs only when it is triggered.

Three levels of triggering exist: always, gentle, loose. Always means that RC will be active once

sum_actual reaches regions 3 or 4. Gentle will trigger RC once sum_actual reaches regions 2 or 5. Loose

waits to trigger RC when sum_actual reaches regions 1 or 6.

RC will deactivate (triggered = false) once sum_actual begins to track sum_target over a series of

macroblocks. Specifically, the sign of the rate of change for bytediff is monitored over a window of

macroblocks. When the sum of these signs over the window falls within a tolerance value (called ‘stable’),

triggered will reset to false.

Panic

When enabled, panic mode will occur whenever sum_actual reaches region 1 and will remain so until

sum_actual reaches region 4. When panicking, all macroblocks will be quantized with QP = MB(n).QP +

QPmax_pos_mod, clamped to 51.

User Controls

This unit achieves a large flexibility by allowing the user to define various key parameters. At the per-

macroblock level, the values of target and max are specified and will create various shapes of curves that

sum_actual will be compared against.

Per-slice, the user can specify the triggering sensitivity and the tolerance required to disable the trigger.

Additionally, the user can enable panic detection.

The point values assigned to each of the 6 regions are exposed to the user which allow for asymmetrical

control for over and under predictions amongst other things. Additionally, the user can specify the initial

values of grow and shrink along with the resistance values applied when correction is invoked.

Lastly, the maximum and minimum values for QPmod are also exposed to the user.

AVC Encoder MBAFF Support

1. Algorithm

Prediction of current macroblock motion vector is possible from neighboring macroblocks

mbAddrA/mbAddrD/mbAddrB/mbAddrC/mbAddrA+1/mbAddrD+1/mbAddrB+1/mbAddrC+1.

The selection of these macroblocks depends on coding type(field/frame) of current macroblock

pair and the coding of neighboring macroblock pair.

Selection of these macroblock pairs is described in detail in following sections.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

1.1 Selection of Top LeftMB pair: The selection of Top Left MB pair depends on coding type of

current and also top left macroblock pair.

1.2 Selection of LeftMB pair: The selection of Left MB pair depends on coding type of current

and also left macroblock pair.

1.3 Selection of Top MB pair: The selection of Top MB pair depends on coding type of current

and also top macroblock pair.

1.4 Selection of Top RightMB pair: The selection of Top Right MB pair depends on coding type

of current and also top right macroblock pair.

1.5 Motion Vector and refIdx Scaling: Motion vectors and reference index of neighboring

macroblocks (mbAddrA/mbAddrB/mbAddrC/mbAddrD) should be scaled before using them into

prediction equations. Again the scaling depends on coding type of current and neighboring

macroblock pair which is described as follows,

 If the current macroblock is a field macroblock and the macroblock mbAddrN is a frame

macroblock ...

mvLXN[ 1 ] = mvLXN[ 1 ] / 2 (8-214)

refIdxLXN = refIdxLXN * 2 (8-215)

 Otherwise, if the current macroblock is a frame macroblock and the macroblock mbAddrN is

a field macroblock ...

mvLXN[ 1 ] = mvLXN[ 1 ] * 2 (8-216)

refIdxLXN = refIdxLXN / 2 (8-217)

 Otherwise, the vertical motion vector component mvLXN[ 1 ] and the reference index

refIdxLXN remain unchanged.

MPEG-2

MPEG2 Common Commands

MFX Commands are organized into groups based on their scope of functioning. There are Pipeline

Common state commands that are common to all codecs (encoder and decoder) and is applicable to the

processing of one full frame/field. There are also individual codec Common state commands that are

common to both encoder and decoder of that particular codec. These latter common state commands,

some are applicable at the processing of one full frame/field, and some are applicable at the processing

of an individual slice level.

MFX_MPEG2_PIC_STATE

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MPEG2 Decoder Commands

These are decoder-only commands. They provide the pointer to the compressed input bitstream to be

decoded.

MFD_MPEG2_BSD_OBJECT

MFD_MPEG2_BSD_OBJECT Inline Data Description

Indirect Data Description

The indirect data start address in MFD_MPEG2_BSD_OBJECT specifies the starting Graphics Memory

address of the bitstream data that follows the slice header. It provides the byte address for the first

macroblock of the slice. Together with the First Macroblock Bit Offset field in the inline data, it provides

the bit location of the macroblock within the compressed bitstream.

The indirect data length in MFD_MPEG2_BSD_OBJECT provides the length in bytes of the bitstream data

for this slice. It includes the first byte of the first macroblock and the last non-zero byte of the last

macroblock in the slice. Specifically, the zero-padding bytes (if present) and the next start-code are

excluded. Hardware ignores the contents after the last non-zero byte. Indirect Data Description illustrates

these parameters for a slice data.

Indirect data buffer for a slice

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

MPEG2 Encoder PAK Commands

The MFC_MPEG2_PAK_INSERT_OBJECT Command is identical to the MFC_AVC_PAK_INSERT_OBJECT

command as described in this document.

The MFC_MPEG2_STITCH_OBJECT Command is identical as MFC_AVC_STITCH_OBJECT command as

described in this document.

MFC_MPEG2_SLICEGROUP_STATE

MFC_MPEG2_PAK_OBJECT

PAK Object Inline Data Description – MPEG-2

The Inline Data includes all the required MB encoding states, constitute part of the Slice Data syntax

elements, MB Header syntax elements and their derivatives. It provides information for the following

operations:

1. Forward and Inverse Transform

2. Forward and Inverse Quantization

3. Advanced Rate Control (QRC)

4. MB Parameter Construction (MPC)

5. VLC encoding

6. Bit stream packing

7. Internal error handling

These state/parameter values may subject to change on a per-MB basis, and must be provided in each

MFC_MPEG2_PAK_OBJECT command. The values set for these variables are retained internally, until they

are reset by hardware Asynchronous Reset or changed by the next MFC_MPEG2_PAK_OBJECT command.

The inline data has been designed to match AVC MB structure for efficient transcoding.

Current MB [x,y] address is not sent, it is assumed that the H/W will keep track of the MB count and

current MB position internally.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

31:27

Reserved: MBZ

22-

MvFormat (Motion Vector Size). This field specifies the size and format of the input motion

vectors.

This field is reserved (MBZ) when the IntraMbFlag = 1.

The valid encodings are:

011 = Unpacked: Two motion vector pairs

Others are reserved.

(The following encodings are intended for other formats:

001 = 1MV: one 16x16 motion vector

010 = 2MV: One 16x16 motion vector pair

011 = 4MV: Four 8x8 motion vectors, or Two 16x8 motion vector pairs

100 = 8MV: Four 8x8 motion vector pairs

101 = 16MV: 16 4x4 motion vectors

110 = 32MV: 16 4x4 motion vector pairs

111 = Packed, number of MVs is given by packedMvNum.)

CbpDcY. This field specifies if the Luma DC coded. Must be 1 for MPEG-2.

CbpDcU. This field specifies if the Chroma Cb DC coded. Must be 1 for MPEG-2.

CbpDcV. This field specifies if the Chroma Cb DC coded. Must be 1 for MPEG-2.

Reserved: MBZ

TransformFlag

Used to indicate transformation type for MPEG-2.

0 = Frame DCT transformation

1 = Field DCT transformation

FieldMbFlag

For MPEG-2, this flag is set to 1 if

either the picture is in field type

or the MB is INTER of field type, i.e. split into two 16x8 field blocks.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

IntraMbFlag

This field specifies whether the current macroblock is an Intra (I) macroblock.

For I-picture MB (IntraPicFlag =1), this field must be set to 1.

This flag must be set in consistent with the interpretation of MbType (inter or intra modes).

0: INTER (inter macroblock)

1: INTRA (intra macroblock)

12:8

MbType

This field is encoded to match with the best macroblock mode determined as described in the next

section. It follows an unified encoding for inter and intra macroblocks according to MFX Encoding

reference as shown in Figure A.

7:3

Reserved : MBZ

SkipMbFlag

By setting it to 1, this field forces an inter macroblock to be encoded as a skipped macroblock. It is

equivalent to mb_skip_flag in AVS spec, Hardware honors input MVs for motion prediction and

forces CBP to zero.

By setting it to 0, an inter macroblock will be coded as a normal inter macroblock. The macroblock

may still be coded as a skipped macroblock, according to the macroblock type conversion rules

described in the later sub sections.

This field can only be set to 1 for certain values of MbType. See details later.

This field is only valid for an inter macroblock. Hardware ignores this field for an intra macroblock.

0 = not a skipped macroblock

1 = is coded as a skipped macroblock

Note: When this flag is set to 1, the correct MVs are assumed for HW decoder to generate decoded

reconstruction frame.

1:0

InterMbMode

This field is provided to carry redundant information as that encoded in MbType.

This field is only valid if IntraMbFlag =0, otherwise, it is ignored by hardware.

31:16

MbYCnt (Vertical Origin). This field specifies the vertical origin of current macroblock in the

destination picture in units of macroblocks.

Format = U16 in unit of macroblock.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

15:0

MbXCnt (Horizontal Origin). This field specifies the horizontal origin of current macroblock in the

destination picture in units of macroblocks.

Format = U16 in unit of macroblock.

31:24

MaxSizeInWord

PAK should not exceed this budget accumulatively, otherwise it will trickle the PANIC mode.

23:16

TargetSizeInWord

PAK should use this budget accumulatively to decide if it needs to limit the number of non-zero

coefficients.

15:13

MBZ

12:0

Cbp – Coded Block Pattern. This field specifies whether blocks are present or not.

Format = 6-bit mask (or 8-bit, & 12-bit, for 422 and 444).

Bit 11: Y0Bit 10: Y1Bit 9: Y2Bit 8: Y3

Bit 7: Cb4Bit 6: Cr5Bits 0-5: MBZ

LastMbInSlice – the last MB in a slice.

FirstMbInSlice – the first slice in a slice, it requires slice header insertion.

29:28

MBZ

EnableCoeffClamp

1 = the magnitude of coefficients of the current MB will be clamped based on the clamping matrix

after quantization

0 = no clamping

LastMbInSG

1 – the current MB is the last MB in the current slice group.

MbSkipConvDisable

This is a per-MB level control to enable and disable skip conversion. This field is ORed with

SkipConvDisable field. This field is only valid for a P or B slice. It must be zero for other slice types.

Rules are provided in Macroblock Type Conversion Rules

0 - Enable skip type conversion for the current macroblock

1 – Disable skip type conversion for the current macroblock

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

FirstMbInSG

1 – the current MB is the last MB in the current slice group.

23:20

MBZ

19:16

MvFieldSelect – Motion Vertical Field Select. A bit-wise representation of a long [2][2] array as

defined in §6.3.17.2 of the ISO/IEC 13818-2 (see also §7.6.4).

Bit

MVector[r]

MVector[s]

MotionVerticalFieldSelect Index

Format = MC_MotionVerticalFieldSelect.

0 = The prediction is taken from the top reference field.

1 = The prediction is taken from the bottom reference field.

15:5

MBZ Reserved

4:0

QpScaleCode

31:16

MV[0][0].y – the y coordinate of the first forward MV

if Mv[0][0] n/a:

if Mv[1][0] available, it MUST be set to the same value as Mv[1][0].

else it MUST be set to the value 0

15:0

MV[0][0].x – the x coordinate of the first forward MV

if Mv[0][0] n/a:

if Mv[1][0] available, it MUST be set to the same value as Mv[1][0].

else it MUST be set to the value 0

31:0

MV[1][0] – the first backward MV

if Mv[1][0] n/a: it MUST be set to the same value as Mv[0][0]

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

31:0

MV[0][1] – the second forward MV

if Mv[0][1] n/a:

if Mv[1][1] available, it MUST be set to the same value as Mv[1][1].

else it MUST be set to the same value as Mv[0][0]

31:0

MV[1][1] – the second backward MV

if Mv[1][1] n/a: it MUST be set to the same value as Mv[1][0]

The mapping between MPEG-2 spec and MfxMbCode can be achieved according to the following:

1) Renamed variables with identical meaning:

MPEG-2 Spec

MFX API

Value

macroblock_quant

MbQuantPresent

0 or 1

macroblock_intra

IntraMbFlag

0 or 1

dct_type

Transform8x8Flag

0 or 1

macroblock_pattern

Cbp8x8

remapped

2) Macroblock type remapping:

B-spec Entry

MPEG-2 Spec

Fra

Typ

Intra

Flag

Ski

Fla

Typ

5Bi

Fiel

Fla

Inte

macroblock_

intra

motion_type

_bit0

motion_type

_bit1

motion_for

ward

motion_back

ward

IPB

Intra

0/1

Skip

01h

02h

03h

0/1

MV*

01h

0/1

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

B-spec Entry

MPEG-2 Spec

Fra

Typ

Intra

Flag

Ski

Fla

Typ

5Bi

Fiel

Fla

Inte

macroblock_

intra

motion_type

_bit0

motion_type

_bit1

motion_for

ward

motion_back

ward

Fra

type

01h

Fra

Fiel

type

04h

Fra

dual

prim

19h

Field

One

16x

01h

Field

Two

16x

04h

Field

dual

prim

19h

Fra

type

01h

02h

03h

Fra

Fiel

type

04h

06h

14h

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

B-spec Entry

MPEG-2 Spec

Fra

Typ

Intra

Flag

Ski

Fla

Typ

5Bi

Fiel

Fla

Inte

macroblock_

intra

motion_type

_bit0

motion_type

_bit1

motion_for

ward

motion_back

ward

Field

One

16x

01h

02h

03h

Field

Two

16x

04h

06h

14h

 Notice that there is no special way to indicate 0 motion vector case for P frame. It is for PAK to

handle internally by checking up the motion vector values.

 Notice also, the MbType5bits is adapted from AVC DXVA macroblock types. It may seems awkward

from MPEG-2 perspective, but provides a common VME interface for us for simpler HW design and

help the advanced transcoding solution.

MFX HW Interface and DXVA Conversion

Map DXVA to HW BSpec

Location

Dword

BSPEC

BYTE

MPEG-2

DXVA

DW0

MbMode

0.0-1

0[0-1]

InterMbMode

see (A)

0.2

0[2]

SkipMbFlag

<-MBskipsFollowing

0.3

0[3]

mbz

0.4-0.5

0[4-5]

IntraMbMode

IntraMacroblock

0.6

0[6]

mbz

0.7

0[7]

FieldMbPolarity

derived

MbType

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Location

Dword

BSPEC

BYTE

MPEG-2

DXVA

1.0-1.4

0[8-12]

MbType5Bits

see (A)

1.5

0[13]

IntraMbFlag

IntraMacroblock

1.6

0[14]

FieldMbFlag

see (A)

1.7

0[15]

TransformFlag

FieldResidual

MbFlag

2.0

0[16]

ResidDataFlag

HostResDiff

2.1

0[17]

CbpDcV

PAK control

2.2

0[18]

CbpDcU

PAK control

2.3

0[19]

CbpDcY

PAK control

2.4-2.6

0[20-22]

MvFormat

= 3, derived

2.7

0[23]

mbz

0[24-31]

PackedMvNum

see (A)

DW1

4-5

1[0-15]

MbXCnt

wMBaddress

6-7

1[16-31]

MbYCnt

wMBaddress

DW2

2[0-7]

bNumCoef[0]

8.0-8.5

2[0-5]

mbz

8.6-8.7

2[6-7]

CbpAcUV

PAK control

2[8-11]

CbpAcY

PAK control

2[12-15]

mbz

2[16-23]

TargetedSzInWord

2[24-31]

MaxSzInWord

DW3

Qscale

derived

12.0-6

3[0-6]

QScaleCode

12.7

3[7]

QScaleType

3[8-15]

mbz

3[16-19]

MvFieldSelect

MvertFieldSel

3[20-23]

mbz

MbExtFlag

15.0

3[24]

mbz

15.1

3[25]

SkipMvConvDisable

15.2

3[26]

LastMbFlag

PAK control

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Location

Dword

BSPEC

BYTE

MPEG-2

DXVA

15.3

3[27]

EnableCoeffClamp

PAK control

15.4-5

3[28-29]

MbScanMethod

MBscanMethod

15.6

3[30]

NewSliceFlag

PAK control

15.7

3[31]

EndSliceFlag

PAK control

DW4-7

16-32

4-7[all]

MV[2][2][2]

MVector[4][2]

(A): Set InterMbMode, MbType5bits, FieldMbFlag, and PackedMvNum from DXVA parameters:

if(IntraMacroblock) return (TYPE_INTRA);

else if(MotionType==3){ // dual prime

MbType5bits = 0x19; FieldMbFlag = 0; InterMbMode = 0; PackedMvNum = 2; return

(DUAL_PRIME);

}

else{

IsFieldFrame = a PicState derivative; switch(MotionType+IsFieldFrame{

case 1: // Two 16x8 field in Frame Frame

case 3: // Two 16x8 field in Field Frame

FieldMbFlag = 1; InterMbMode= 1; switch(MotionForward |Motionbackward «1)){

case 1:

MbType5bits = 4; PackedMvNum = 2; break;

case 2:

MbType5bits = 6; PackedMvNum = 2; break;

case 3:

MbType5bits = 0x14; PackedMvNum = 4; break;

}

break;

case 2: // 16x16 block in either case

FieldMbFlag = IsFieldFrame; InterMbMode = 0;

switch(MotionForward|(Motionbackward«1)){

case 1:

MbType5bits = 1; PackedMvNum = 1; break;

case 2:

MbType5bits = 2; PackedMvNum = 1; break;

case 3:

MbType5bits = 3; PackedMvNum = 2; break;

}

break;

}

Map HW Bspec to DXVA

Location

BSPEC

BYTE

DXVA

MPEG-2

0-1

wMBaddress

= MbYCnt*MbW + MbXCnt

2-3

wMBtype

2.0

IntraMacroblock

= IntraMbFlag

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Location

BSPEC

BYTE

DXVA

MPEG-2

2.1

MotionForward

see (B)

2.2

MotionBackward

see (B)

2.3

Motion4MV

VC-1 only, MBZ for Mpeg-2

2.4

Reserved

2.5

FieldResidual

= TranformFlag

2.6-2.7

MBscanMethod

= MbScanMethod

3.0-3.1

MotionType

see (B)

3.2

HostResDiff

= ResidDataFlag

3.3

Reserved

3.4-3.7

MvertFieldSel

= MvFieldSelect

MBskipsFollowing

count SkipMbFlag

5-7

MBdataLocation

n/a

8-9

wPatternCode

= CbpAcY|UV

10-15

bNumCoef[6]

n/a

16-32

MVector[4][2]

= MV[2][2][2]

(B): Set MBtype and MotionType from Bspec interface

if(MbIntraFlag) return (TYPE_INTRA);

else {

if(MbType5Bits&8) { // dual prime

MotionForward = 1;

MotionBackward = 0;

MotionType = 3;

return (DUAL_PRIME);

}

else {

// redundant: InterMbMode = !!(MbType5Bits&4);

if(InterMbMode) {

MotionForward = !(MbType5Bits&2);

MotionBackward = !!(MbType5Bits&0x12);

}

else {

MotionForward = (MbType5Bits&1);

MotionBackward = !!(MbType5Bits&2);

}

MotionType = 2-(InterMbMode^FieldMbFlag);

// equivalently the 2 bits are:

// MotionType0 = (InterMbMode^FieldMbFlag);

// MotionType1 = ~MotionType0;

return (TYPE_INTER);

}

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Video Codec VC-1

This section describes support for the open video compression standard VC-1, which is the common

name for SMPTE 421M approved on April 3, 2006.

VC1 Common Commands

MFX Commands are organized into groups based on their scope of functioning. There are Pipeline

Common state commands that are common to all codecs (encoder and decoder) and is applicable to the

processing of one full frame/field. There are also individual codec Common state commands that are

common to both encoder and decoder of that particular codec. These latter common state commands,

some are applicable at the processing of one full frame/field, and some are applicable at the processing

of an individual slice level.

MFX_VC1_PRED_PIPE_STATE

MFX_VC1_DIRECTMODE_STATE

VC1 Decoder Commands

These are decoder-only commands. They provide the pointer to the compressed input bitstream to be

decoded.

MFD_VC1_LONG_PIC_STATE

AltPQuantConfig and AltPQuantEdgeMask are derived based on the following variables: DQUANT,

DQUANTFRM, DQPROFILE, DQSBEDGE, DQDBEDGE, and DQBILEVEL defined in the VC1 standard, as

shown in the following table.

Definition of AltPQuantConfig and AltPQuantEdgeMask

Inputs

Outputs

Description

DQUANT

FRM

PROFILE

DQDB

EDGE

DQSB

EDGE

DQBI

LEVEL

AltPQuant

Config

AltPQuant

EdgeMask

00b

0000b

No AltPQuant

00b

0000b

No AltPQuant

11b

10b

0000b

All MBs are different with

MQDIFF and ABSMQ

11b

0000b

All MBs may switch with 1-bit

MQDIFF

01b

1111b

All edge MBs

00b

01b

1111b

All edge MBs

01b

00b

01b

0011b

Left and top MBs

01b

0110b

Top and right MBs

01b

10b

01b

1100b

Right and bottom MBs

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Inputs

Outputs

Description

DQUANT

FRM

PROFILE

DQDB

EDGE

DQSB

EDGE

DQBI

LEVEL

AltPQuant

Config

AltPQuant

EdgeMask

01b

11b

01b

1001b

Bottom and left MBs

10b

00b

01b

0001b

Left MBs

10b

01b

0010b

Top MBs

10b

01b

0100b

Right MBs

10b

11b

01b

1000b

Bottom MBs

MFD_VC1_SHORT_PIC_STATE

Intel HW does not use the MVMODE and MVMODE2 provided at the revised DXVA2 VC1 VLD interface,

instead, HW will decode them directly from the bitstream picture header.

MFD_VC1_BSD_OBJECT

For VC1, a slice/picture is always started with MB x positon equal to 0. Hence, no need to include in the

Object Command.

Handling Emulation Bytes

In general, VC1 BSD unit is capable of handling emulation prevention bytes. However, there is a corner

case that requires host software’s intervention. Host software needs to overwrite the emulation byte if it

overlaps the macroblock layer decode and there is not enough information for the hardware to detect

the emulation byte.

The emulation bytes might have an overlap between the picture states and the first macroblock data. If

the emulation bytes are 0x00 0x000x03 0x00 and the macroblock data starts in the middle of byte1

(0x00), then the host software needs to overwrite the 0x03 byte location with the previous byte (0x00)

and change the byte offset accordingly. The hardware wouldn’t know what the 1

byte was and will miss

this 0x03 removal.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

VP8

MFX_VP8_STATISTICS - Encoder Only

Address

offset

Name

Description

31:0

32-bit

P0 Partition Bit Size

// Raw bit count per Partition

No Fulsim

Validation

31:0

32-bit

P1 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P2 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P3 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P4 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P5 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P6 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P7 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P8 Partition Bit Size

No Fulsim

Validation

31:0

32-bit

P1-8 Partition Bit

Size Sum

// This is raw BIT sum of Partition1-8, Not Byte

sum.

No Fulsim

Validation

No Fulsim

Validation

31:0

32-bit

Segment0 Total Bit

Count

// Raw bit count per Segment

No Fulsim

Validation

31:0

32-bit

Segment1 Total Bit

Count

No Fulsim

Validation

31:0

32-bit

Segment2 Total Bit

Count

No Fulsim

Validation

31:0

32-bit

Segment3 Total Bit

Count

No Fulsim

Validation

15:0

16-bit

Segment0 Num of

// Num of MB per Segment

No Fulsim

Validation

31:16

16-bit

Segment1 Num of

No Fulsim

Validation

Media VDBOX

100

Doc Ref # IHD-OS-KBL-Vol 8-1.17

15:0

16-bit

Segment2 Num of

No Fulsim

Validation

31:16

16-bit

Segment3 Num of

No Fulsim

Validation

31:0

32-bit

P0 Partition Bit Size

before CPBAC

// Bin Count of Syntax Element before CPBAC

No Fulsim

Validation

31:0

32-bit

P1 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P2 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P3 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P4 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P5 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P6 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P7 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

P8 Partition Bit Size

before CPBAC

No Fulsim

Validation

31:0

32-bit

Reserved

31:0

32-bit

Reserved

31:0

32-bit

Reserved

31:0

32-bit

Reserved

31:0

32-bit

Reserved

31:0

32-bit

Reserved

31:0

32-bit

Reserved

31:0

32-bit

mb skip prob Total

cnt

Total Number of MB when MBNoCoeffSkip ==

1(if MBNoCoeffSkip == 0, this field =0) Please

see foot notes 1.

31:0

32-bit

mb skip prob cnt

Number of MB with MBSkip == 1 when

MBNoCoeffSkip == 1. Please see foot notes 1.

303-34

15:0

token_statistic count

Token Statistics Counters. Please see foot notes

31:16

Reserved

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

101

Programming Note

Context:

MFX_VP8_STATISTICS - Encoder Only

1. MBSkip_Prob_Total_Cnt and MBSkip_Prob_Cnt are collected to generate MBSkipProbability. After bit

packing, Hardware returns both MBSkip_Prob_Total_Cnt and MBSkip_Prob_Cnt. Optimal packing could be

performed in subsequent pass using MBSkipProbability of

round (256 * (MBSkip_Prob_Total_Cnt- MBSkip_Prob_Cnt) / MBSkip_Prob_Total_Cnt))

2. Token Statistics counters collects token statistics of particular plane (4), coefficient band(8), neighbor

context(3) and tree node position(11) as described in WebM Spec. Out of the space of 1056 counters, there

are 270 of which has high significant in compression efficiency and are chosen for statistics collection.

VP8 Common Commands

Following are VP8 Common Commands:

MFX_VP8_PIC_STATE

For VP8 HW PAK, there are four VP8 versions supported and their programming is shown in Table1

below.

Version

MC Filter Select

Chroma Full Pixel MC Filter Mode

DBLK FilterType

DBLK FilterLevel for Segment0

Any FilterLevel

Table1: VP8 Version

MC Filter Select: MFX_VP8_PIC_STATE.DW2.Bit0

Chroma Full Pixel MC Filter Mode: MFX_VP8_PIC_STATE.DW2.Bit1

DBKL Filter Type: MFX_VP8_PIC_STATE.DW2.Bit4

DBLK Filter Level for Segment0: MFX_VP8_PIC_STATE.DW3.Bit5:0

1. Note that when multiple segment is enabled, if Segment0 DBLK Filter is programmed to 0,

Segment1,2,3 DBLK Filter should be set to 0 as well.

2. Note that MFX_VP8_Encoder_CFG.BW22.Bit22:20 (Bitstream Format Version). This field is used for

generating Uncompressed header only. It is not used to control any Filter.

Media VDBOX

102

Doc Ref # IHD-OS-KBL-Vol 8-1.17

VP8 Decoder Commands

Following are VP* Decoder Commands:

MFD_VP8_BSD_OBJECT

VP8 Encoder Commands

MFX_VP8_Encoder_CFG

MFX_VP8_BSP_BUF_BASE_ADDR_STATE

MFX_VP8_PAK_OBJECT

VP8 PAK Object inline data:

Inline Data Description - VP8 PAK OBJECT

Y_Mode for macroblock in non-B mode

DC_PRED

V_PRED

H_PRED

TM_PRED

B_PRED

NEARESTMV

NEARMV

ZEROMV

NEWMV

SPLITMV

2 B mode

B_DC_PRED

B_TM_PRED

B_VE_PRED

B_HE_PRED

B_LD_PRED

B_RD_PRED

B_VR_PRED

B_VL_PRED

B_HD_PRED

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

103

B_HU_PRED

SPLIT_LEFT

SPLIT_ABOVE

SPLIT_ZERO

SPLIT_NEW

JPEG and MJPEG

JPEG Decoder Commands

Following are JPEG Decoder Commands:

MFD_JPEG_BSD_OBJECT

MFX_JPEG_PIC_STATE command is used for both encoding and decoding. Note the duplicate bits and

the "Exists If" rows that specify what the bits represent for Encoder and Decoder.

MFX_JPEG_PIC_STATE

For JPEG decoding, the following program note is informative.

For Rotation, it is important to note that rotation of 90 or 270 degrees also requires exchanging

FrameWidthlnBlksMinus1 with FrameHeightlnBlksMinus1 in the command. In addition, the rotation

of 90 or 270 degrees also requires transportation of the quantization matrix will be transposed into the

position (y, x).

Chroma type is determined by the values of horizontal and vertical sampling factors of the components

(Hi and Vi where i is a component id) in the Frame header as shown in the following table.

0: YUV400

Not available

1: YUV420

2: YUV422H_2Y

3: YUV444

4: YUV411

5: YUV422V_2Y

6: YUV422H_4Y

7: YUV422V_4Y

For YUV400, the value of V1 can be 1, 2, or 3 and will be same as the value of H1, and the Minimum

coded unit (MCU) is one 8x8 block. For the other chroma formats, if non-interleaved data, the MCU is

one 8x8 block. For interleaved data, the MCU is the sequence of block units defined by the sampling

factors of the components.

For example, the following figures show the MCU structures of interleaved data and the decoding order

of blocks in the MCU:

Media VDBOX

104

Doc Ref # IHD-OS-KBL-Vol 8-1.17

422H_2Y

422H_4Y

422V_2Y

422V_4Y

If picture width X in the Frame header is not a multiple of 8, the decoding process needs to extend the

number of columns to complete the right-most sample blocks. If the component is to be interleaved, the

decoding process needs to extend the number of samples by one or more additional blocks so that the

number of blocks is an integer multiple of Hi. In other words, “The number of blocks in width” in the

table should be an integer multiple of (8xH1). Similarly, if picture height Y in the Frame header is not a

multiple of 8, the decoding process needs to extend the number of lines to complete bottom-most

block-row. If the component is to be interleaved, the decoding process also needs to extend the number

of lines by one or more additional block-rows so that the number of block-row is an integer multiple of

(8xV1). For example, if non-interleaved YUV411 with X=270, then “The number of blocks in width” shall

be (270 + 7) / 8 = 34, where “/” is integer division. Therefore, FrameWidthlnBlksMinus1 is set to 33.

However, for interleaved data, “The number of blocks in width” shall be ((270 + 31) / 32) x 4 = 36.

Therefore, FrameWidthlnBlksMinus1 is set to 35.

VertUpSamplingEnb is used to convert an input chroma420 to an output chroma422 in the surface

format of YUY2 or UYVY. To enable this flag, the input should be interleaved Scan, InputFormatYUV

should be set to YUV420, and OutputFormatYUV should be set to YUY2 or UYVY. Vertical 2:1 up-

sampling is only applied to chroma blocks where each line of 8x8 block pixels is replicated to make 8x16

U/V blocks. For example:

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

105

VertDownSamplingEnb is used to convert an input chroma422 to an output chroma420 in the surface

format NV12. To enable this flag, the input should be interleaved Scan, InputFormatYUV should be set to

YUV422H_2Y or YUV422H_4Y, and OutputFormatYUV should be set to NV12. Combined with

AvgDownSampling flag, the following table and figures show the down-sampling methods.

VertDownSamplingEnb

AvgDownSampling

Down-Sampling Methods

0 or 1

No down-sampling.

Drop every other line:

Average vertically neighboring two pixels:

Media VDBOX

106

Doc Ref # IHD-OS-KBL-Vol 8-1.17

VertDownSamplingEnb

AvgDownSampling

Down-Sampling Methods

The recent history forJPEG Decoder Commands are described in the following:



 InputFormat is the same as IVB, and should be programmed the same way as IVB.

 If the InputFormat is YUV400 or YUV444 or YUV411, then output cannot be NV12, YUY2 or UYVY, it

has to be planar (like legacy IVB). But for 420 and 422 InputFormat, there’s a choice of having

Planar, NV12, YUY2 or UYVY OutputFormat and the surface state should be programmed

accordingly.

 Refer “Output Format YUV” field for more details.

MFX_JPEG_HUFF_TABLE_STATE

JPEG Encoder Commands

JPEG Encoder Command Sequence:

MFX_PIPE_MODE_SELECT

MFX_SURFACE_STATE

MFX_PIPE_BUF_ADDR_STATE

MFX_JPEG_PIC_STATE

MFX_FQM_STATE ( One each for Luma, CB and CR)

MFC_JPEG_ HUFF_TABLE_STATE ( Huffman table 0 and 1 need two commands to be issued).

MFC_JPEG_SCAN_OBJECT

MFX_PAK_INSERT_OBJECT ( Multiple commands can be given based on the need)

Following are JPEG Encoder Commands:

MFX_JPEG_PIC_STATE command is used for both encoding and decoding. Note the duplicate bits and

the "Exists If" rows that specify what the bits represent for Encoder and Decoder.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

107

MFX_JPEG_PIC_STATE

Programming Note: For completion of partial MCUs in JPEG encoding, it is important to note the

following:

If the image’s dimensions are not an exact multiple of the MCU size, the encoded data should include

padding to round up to the next complete MCU, which is called completion of partial MCU. If the

number of lines is not aligned with MCU structure (not a multiple of MCU size, i.e. 8, 16, 32), the

encoding process needs to extend the number of lines to complete the bottom-most MCU-row.

Similarly, if the number of samples per line is not aligned with MCU structure, the encoding process

needs to extend the number of columns to complete the right-most sample MCUs. JPEG standard

recommends that any incomplete MCUs be completed by replication of the right-most column and the

bottom line of each component Y, U, and V.

The following equations are used to set the command for encoding partial MCUs.

FrameWidthlnBlksMinus1 = (((X + (H

*8 -1)) / (H

*8)) * H

) – 1

FrameHeightlnBlksMinus1 = (((Y + (V

*8 -1)) / (V

*8)) * V

) – 1

For YUV400,

PixelsInHoriLastMCU = X % 8

PixelsInVertLastMCU = Y % 8

For YUV420,

PixelsInHoriLastMCU = X % 16 if X % 2 = 0, ((X % 16) + 1) % 16 if X % 2 = 1

PixelsInVertLastMCU = Y % 16 if Y % 2 = 0, ((Y % 16) + 1) % 16 if Y % 2 = 1

For YUV422H_2Y,

PixelsInHoriLastMCU = X % 16 if X % 2 = 0, ((X % 16) + 1) % 16 if X % 2 = 1

PixelsInVertLastMCU = Y % 8

X: the number of samples per line in Y-image

Y: the number of lines in Y-image

H1: horizontal sampling factor of Y-image in the Frame header

V1: vertical sampling factor of Y-image in the Frame header

Note that PixelsInHoriLastMCU=0 does not mean the num of pixels in the right-most MCUs = 0, but

does mean that the right-most MCUs are fully filled with pixels, i.e., not a partial MCU.

For example, for input image dimension 17x26 pixels and an interleaved Scan, the following equations

and the table show how to set the command for each OutputMcuStructure.

YUV400

YUV420

YUV422H_2Y

MCU size of Y

8x8

16x16

16x8

MCU size of U and V

8x8

and V

1, 1

2, 2

2, 1

Media VDBOX

108

Doc Ref # IHD-OS-KBL-Vol 8-1.17

YUV400

YUV420

YUV422H_2Y

FrameWidthlnBlksMinus1

FrameHeightlnBlksMinus1

PixelsInHoriLastMCU

PixelsInVertLastMCU

MFC_JPEG_SCAN_OBJECT

MFC_JPEG_ HUFF_TABLE_STATE

The JPEG standard Table K.5 shows the real table of code length and code word as follows:

Run/Size

Code length

Code word

0/0 (EOB)

1010

0/1

0/2

0/3

100

0/4

1011

0/5

11010

0/6

1111000

0/7

11111000

0/8

1111110110

0/9

1111111110000010

0/A

1111111110000011

It is not necessary to send Run/size in the command as driver will send the increasing order of run/size.

Each symbol aligns to a DWord with the following byte structure. Each DWord (a group of 4 bytes)

contains Byte0 for Code length, Byte1 and Byte2 for Code word, and Byte3 for dummy.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

109

Driver will program to always send 12 pairs of Code length and Code Word in DC coefficient table and

162 pairs in AC coefficient table. When a Huffman table contains valid full entries of Run/Size, all the

Code word and Code length will not be zero. If a Huffman table is customized or optimized, the table can

contain smaller set of Code length and Code Word, i.e., the number of entries of the real Huffman table

will be less than 12 for DC, or less than 162 for AC. For the customized Huffman table, driver will set the

missing entry (Run/Size) to Code length = 0 and Code word = 0.

MFX_PAK_INSERT_OBJECT

More Decoder and Encoder

MFD IT Mode Decode Commands

This is the decoder-only command to support the IT-mode specified in DXVA interface.

Only MPEG2-IT mode is supported.

MFD_IT_OBJECT

Common Indirect IT-COEFF Data Structure

Transform-domain residual data block in MPEG2-IT mode follows the same data structure.

The indirect IT-COEFF data start address in MFD_IT_OBJECT command specifies the doubleword aligned

address of the first non-zero DCT coefficient of the first block of the macroblock. Only the non-zero

coefficients are present in the data buffer and they are packed in the 8x8 block sequence of Y0, Y1, Y2,

Y3, Cb4 and Cr5, as shown in Common Indirect IT-COEFF Data Structure. When an 8x8 block is further

subdivided into 4x4 subblocks, the coefficients, if present, are organized in the subblock order. The

smallest subblock division is referred to as a transform block. The indirect IT-COEFF data length in the

command includes all the non-zero coefficients for the macroblock. It must be doubleword aligned.

Structure of the IDCT Compressed Data Buffer

Each non-zero coefficient in the indirect data buffer is contained in a doubleword-size data structure

consisting of the coefficient index, end of block (EOB) flag and the fixed-point coefficient value in 2’s

compliment form. As shown in Common Indirect IT-COEFF Data Structure, index is the row major 'raster'

index of the coefficient within a transform block (please note that it is not converted to 8x8 block basis).

A coefficient is a 16-bit value in 2's complement.

Media VDBOX

110

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Structure of a transform-domain residue unit

DWord

Bit

Description

31:16

Transform-Domain Residual (coefficient) Value. This field contains the value of the non-zero

transform-domain residual data in 2’s compliment.

15:7

Reserved: MBZ

6:1

Index. This field specifies the raster-scan address (raw address) of the coefficient within the

transform block. For a coefficient at Cartesian location (row, column) = (y, x) in a transform block of

width W, Index is equal to (y * W + x). For example, coefficient at location (row, column) = (0, 0) in a

4x4 transform block has an index of 0; that at (2, 3) has an index of 2*4 + 3 = 11.

The valid range of this field depends on the size of the transform block.

Format = U6

Range = [0, 63]

EOB (End of Block). This field indicates whether the transform-domain residue is the last one of

the current transform block.

Allowed transform block dimensions per coding standard

Transform Block Dimension

MPEG2

8x8

Yes

8x4

4x8

4x4

Inline Data Description in MPEG2-IT Mode

The content in this command is similar to that in the MEDIA_OBJECT command in IS mode described in

the Media Chapter.

Each MFD_IT_OBJECT command corresponds to the processing of one macroblock. Macroblock

parameters are passed in as inline data and the non-zero DCT coefficient data for the macroblock is

passed in as indirect data.

Inline Data Description in MPEG2-IT Mode depicts the inline data format. Inline data starts at dword 7 of

MFD_IT_OBJECT command. There are 7 dwords total.

Inline data in MPEG2-IT Mode

DWord

Bit

Description

31:28

Motion Vertical Field Select. A bit-wise representation of a long [2][2] array as defined in §6.3.17.2

of the ISO/IEC 13818-2 (see also §7.6.4).

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

111

DWord

Bit

Description

Bit

MVector[r]

MVector[s]

MotionVerticalFieldSelect Index

Format = MC_MotionVerticalFieldSelect.

0 = The prediction is taken from the top reference field.

1 = The prediction is taken from the bottom reference field.

Reserved (was Second Field)

Reserved. (HWMC mode)

25:24

Motion Type. When combined with the destination picture type (field or frame) this Motion Type

field indicates the type of motion to be applied to the macroblock. See ISO/IEC 13818-2 §6.3.17.1,

Tables 6-17, 6-18. In particular, the device supports dual-prime motion prediction (11) in both

frame and field picture type.

Format = MC_MotionType

Value

Destination = Frame

Picture_Structure = 11

Destination = Field

Picture_Structure != 11

‘00’

Reserved

‘01’

Field

‘10’

Frame

16x8

‘11’

Dual-Prime

23:22

Reserved. (Scan method)

DCT Type. This field specifies the DCT type of the current macroblock. The kernel should ignore this

field when processing Cb/Cr data. See ISO/IEC 13818-2 §6.3.17.1. This field is zero if Coded Block

Pattern is also zero (no coded blocks present).

0 = MC_FRAME_DCT (Macroblock is frame DCT coded).

1 = MC_FIELD_DCT (Macroblock is field DCT coded).

Reserved (was Overlap Transform - H261 Loop Filter).

Reserved (was 4MV Mode - H263)

Macroblock Motion Backward. This field specifies if the backward motion vector is active. See

ISO/IEC 13818-2 Tables B-2 through B-4.

0 = No backward motion vector.

1 = Use backward motion vector(s).

Media VDBOX

112

Doc Ref # IHD-OS-KBL-Vol 8-1.17

DWord

Bit

Description

Macroblock Motion Forward. This field specifies if the forward motion vector is active. See ISO/IEC

13818-2 Tables B-2 through B-4.

0 = No forward motion vector.

1 = Use forward motion vector(s).

Macroblock Intra Type. This field specifies if the current macroblock is intra-coded. When set,

Coded Block Pattern is ignored and no prediction is performed (i.e., no motion vectors are used).

See ISO/IEC 13818-2 Tables B-2 through B-4.

0 = Non-intra macroblock.

1 = Intra macroblock.

15:12

Reserved : MBZ

11:6

Coded Block Pattern. This field specifies whether blocks are present or not.

Format = 6-bit mask.

Bit 11: Y0

Bit 10: Y1

Bit 9: Y2

Bit 8: Y3

Bit 7: Cb4

Bit 6: Cr5

5:4

Reserved. (Quantization Scale Code)

LastMBInRow – This field indicates the last MB in each row.

2:0

Reserved: MBZ

31:16

Reserved : MBZ

15:8

VertOrigin - Vertical Origin

In unit of macroblocks relative to the current picture (frame or field).

7:0

HorzOrigin - Horizontal Origin

In unit of macroblocks.

31:16

Motion Vectors – Field 0, Forward, Vertical Component. Each vector component is a 16-bit

two’s-complement value. The vector is relative to the current macroblock location. According to

ISO/IEC 13818-2 Table 8, the valid range of each vector component is [-2048, +2047.5], implying a

format of s11.1. However, it should be noted that motion vector values are sign extended to 16 bits.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

113

DWord

Bit

Description

15:0

Motion Vectors – Field 0, Forward, Horizontal Component

31:16

Motion Vectors – Field 0, Backward, Vertical Component

15:0

Motion Vectors – Field 0, Backward, Horizontal Component

31:16

Motion Vectors – Field 1, Forward, Vertical Component

15:0

Motion Vectors – Field 1, Forward, Horizontal Component

31:16

Motion Vectors – Field 1, Backward, Vertical Component

15:0

Motion Vectors – Field 1, Backward, Horizontal Component

Indirect Data Format in MPEG2-IT Mode

MPEG2-IT mode only contains IT-COEFF indirect data which is described in Section Common Indirect IT-

COEFF Data Structure.

MFX Deblocking Commands

Following are MFX Deblocking Commands:

MFX_DBK_OBJECT

Media VDBOX

114

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Encoder StreamOut Mode Data Structure Definition

When StreamOut is enabled, per MB (and/or per Slice, per Picture) intermediated coding data (for

example, bit allocated for each MB, and so on) are sent to the memory in a fixed record format (and of

fixed size) from the PAK. The per-MB records must be written in a strict raster order and with no gap

(that is, every MB regardless of its mb_type and slice type, must have an entry in the StreamOut buffer).

Therefore, the consumer of the StreamOut data can offset into the StreamOut Buffer (StreamOut Data

Destination Base Address) using individual MB addresses.

Adding per macroblock stream out for PAK is for the following purposes:

 Immediate multi-pass PAK (without host or EU intervention)

 3200-bit conformance

 Re-quantization

 Providing information for host for offline processing

 Providing information for updated QP’s

The description for the fixed format PAK streamout record:

Streamout Pointer: Use the existing streamout pointer and enabler

Per Macroblock Information (a fixed size structure)

DWord

Bit

Description

31:24

MbQpY - Actual QPY used by the macroblock.

23:16

MbClock16 – MB compute clocks in 16-clock unit.

15:8

Reserved: MBZ

7:4

Reserved: MBZ (future conformance flags)

Reserved

MbRcFlag: MB level Rate control flag(pass through)

The same value as RateControlCounterEnable of MFX_AVC_SLICE_STATE Command

MbInterConfFlag: MB level InterMB conformance flag to trigger mutli-pass

1- if total Bit Count of an inter macroblock is more than Inter Conformance Max size limit in the

MFX_AVC_IMG_STATE Command

MbIntraConfFlag: MB level IntraMB conformance flag to trigger mutli-pass

1- if total Bit Count of an intra macroblock is more than Intra Conformance Max size limit in the

MFX_AVC_IMG_STATE Command

31:29

Reserved

28:16

MbBits: Total Bit Count for the macroblock

15:12

Reserved

12:0

MbHdrBits: Header Bit count (bit count due to Pre-coefficient data) for the macroblock

31:27

Reserved

26:0

Cbp: Coded Block Pattern of sub-blocks

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

115

DWord

Bit

Description

31:30

Reserved

IntraMBFlag

28:24

MBType5Bits

23:17

Reserved

ClampFlag: Coefficient clamping flag for RC (Status)

1 - Indicates if clamping of any coefficient is done on the macroblock for Rate Control

15:0

Reserved (future QRC stat output)

PAK Frame Statistics StreamOut

The following frame statistics are written to memory at the conclusion of a frame. If Multipass occurs,

these values are overwritten by the end of any subsequent passes of the current frame (hence it contains

only the final pass statistics).

The streamout is done to the MB streamout surface, starting at the next CL boundary. If MB streamout is

disabled, Frame level streamout starts with 0 offset.

MFX_PAK_FRAME_STATISTICS

Source:

VideoCS

Length

Bias:

DWord

Bit

Description

31:16

Reserved : MBZ

15:0

SumSliceHeader – Report the total size (in bits) of all slice headers inserted into the bitstream for

this frame.

31:0

SumMBHeader – Report the total size (in bits) of all MB headers (non coeff bits) inserted into the

bitstream for this frame.

31:0

SumNZC – Report the total number of nonzero coefficients after quantization.

31:0

Reserved: MBZ

31:16

IntraMB16x16 – Count of # of MB’s that were of type Intra 16x16

15:0

IntraMB8x8 – Count of # of MB’s that were of type Intra 8x8

31:16

IntraMB4x4 – Count of # of MB’s that were of type Intra 4x4

15:0

InterMB16x16 – Count of # of MB’s that were of type Inter 16x16

31:16

InterMB16x8 – Count of # of MB’s that were of type Inter 16x8

15:0

InterMB8x16 – Count of # of MB’s that were of type Inter 8x16

31:16

InterMB8x8 – Count of # of MB’s that were of type Inter 8x8

15:0

InterSkip16x16 – Count of # of MB’s that were of type Inter 16x16 skip

Media VDBOX

116

Doc Ref # IHD-OS-KBL-Vol 8-1.17

8:49

31:0

RhoDomainStats – Each DW contains 1 of the 42 registers containing the raw Rho Domain

coefficient metrics. DW 8 is QP 10 and DW 49 is QP51.

31:0

Reserved: MBZ

PAK Multi-Pass

Multi-Pass PAK Usages:

 Intra MB 3200-bit conformance

 Inter MB Re-quantization

 Frame level Re-quantization

How to Enable Multi-Pass PAK?

 Using the existing conditional batch buffer execution capability to skip/execute the second pass

o How to dynamically change the condition?

 Defined one error condition register with a mask. Do HW status page update at

the end of the first pass. 0 means all OK, non-zero means there is an error

condition, requiring second pass. Mask is used by the host to control what kind of

multi-pass is intended.

 For example, one error bit is 3200-bit conformance violation. Another error bit is

the total bit count exceeds (too much or too little) the target range (need to define

the target range in the state).

 The logic perfectly fits in the conditional batch buffer control logic that VCS

has today in GT. There is no additional logic need to be added in VCS to

support media functionality. (Batch Buffer Skip: This field only takes effect if

Compare Semaphore is set and the value at Semaphore Address is NOT greater

than the Semaphore Data Dword).

 Adding a picture level state command to enable and control the behavior of the second pass PAK

o How to control the re-PAK? Added 3 conformance flags (error registers) in the per-MB

streamout. Then the error control is based on the error register and the mask defined in

picture level states. There are 8 register flags defined out of which only the 3200-bit case

has usage model defined for today. The rest are left for future usage.

Issues and Limitations:

 There is no programmable engine in MFX for flexible control: Therefore, whatever we have defined

must consider flexibility

Following 2 MI packets are used inside VCS without any change to support Multipass-PAK behaviour.

 MI_Conditional_Batch_Buffer_End

 Memory Interface Registers

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

117

Driver Usage

Driver places Image states in one batch buffer and all slice level and macroblock level states into another

batch buffer and link 2 batch buffers. Also replicate Image states with multipass changes in another

batch buffer link them to slice/macroblock batch buffer. In this way, only Image states are replicated but

not the slice/macroblock states. The image states includes all buffers defined at image(indirectMV,

original pixel buffer, etc). Following changes are needed in the Multipass Image State,

 Reset- Stream-Out Enable(disable stream out in the second pass)

 Set- MacroblockStatEnable (enable reading of macroblock status buffer)

 Reset- 3200-bit conformance (do not report 3200-bit conformance)

Define Conditional Batch Buffer End for CS/VCSVINunit

Programming Reference

Monochrome Picture Processing

Monochrome picture is specified using the Surface State with Surface Format of 12. Therefore, MFX

hardware, in either decode or encode mode, does not generate any read or write traffic for U/V

components. The motivation for this bandwidth optimization is that monochrome video coding might be

used for wireless display.

For Encoder:

1. No read in UV original components

2. Processing UV component - no

3. Reconstructed UV component reference picture - no

4. Filter UV component - no

For Decoder:

1. VLD mode: There is no color component coming out of the decoding pipeline in Monochrome

mode and so no processing and not writing output.

Media VDBOX

118

Doc Ref # IHD-OS-KBL-Vol 8-1.17

2. IT mode: There is no color component in the coefficient buffer, and so no processing and not

writing output.

Context Switch

There is no pre-emption for the BCS pipeline; hence every command buffer is required to contain all the

states setup (preamble). Specifically, CPU can not interrupt the BCS-BSD pipe, to stop the operation in

the middle of decoding a bitstream data.

Switch of contexts can only be performed at picture boundary.

No state needs to be saved.

PMSI Support

Pipeline Flush

Implicit flush for AVC and VC1 is performed at the end of Slice : for MPEG2 is done when a new

image/picture command is issued. Because MPEG2 a slice can be one MB, no point to flush. MPEG2 will

snoop the next command if it is an img_state command.

Explicit flush MI (1 bit to do media pipeline vs Gx pipeline) flush and cache flush (switch reference frame)

– MI flush has bit to do cache flush. MI flush is for driver synchronization.

MMIO Interface

A set of registers are defined and accessible through MMIO interface to serve multiple purposes:

 Use for system configuration

 For accessing Performance counters

The following is the table for all the MMIO addresses for MFX.

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

119

Decoder Registers

Following are Decoder Registers:

MFD_ERROR_STATUS - MFD Error Status

AVC CAVLC

AVC CABAC

VC1

MPEG2

JPEG

MFD_PICTURE_PARAM - MFD Picture Parameter

MFX_STATUS_FLAGS - MFX Pipeline Status Flags

MFX_FRAME_PERFORMANCE_CT - MFX Frame Performance Count

MFX_SLICE_PERFORM_CT - MFX Slice Performance Count

MFX_MB_COUNT - MFX Frame Macroblock Count

MFX_SE-BIN_CT - MFX Frame BitStream SE/BIN Count

MFX_LAT_CT1 - MFX_Memory_Latency_Count1

MFX_LAT_CT2 - MFX Memory Latency Count2

MFX_LAT_CT3 - MFX Memory Latency Count3

MFX_LAT_CT4 - MFX Memory Latency Count4

MFX_SE-BIN_CT - MFX Frame BitStream SE/BIN Count

MFX_READ_CT - MFX Frame Motion Comp Read Count

MFX_MISS_CT - MFX Frame Motion Comp Miss Count

Media VDBOX

120

Doc Ref # IHD-OS-KBL-Vol 8-1.17

Encoder Registers

Following are the Encoder Registers:

MFC_VIN_AVD_ERROR_CNTR - MFC_AVC Bitstream Decoding Front-End Parsing Logic Error Counter .

MFC_BITSTREAM_BYTECOUNT_FRAME - Reported Bitstream Output Byte Count per Frame Register

MFC_BITSTREAM_SE_BITCOUNT_FRAME - Reported Bitstream Output Bit Count for Syntax Elements Only

MFC_AVC_CABAC_BIN_COUNT_FRAME - Reported Bitstream Output CABAC Bin Count Register

AVC_CABAC_INSERTION_COUNT - MFC_AVC_CABAC_INSERTION_COUNT

MFC_AVC_MINSIZE_PADDING_COUNT - Bitstream Output Minimal Size Padding Count Report Register

MFC_IMAGE_STATUS_MASK - MFC Image Status Mask

MFC_IMAGE_STATUS_CONTROL - MFC Image Status Control

MFC_QUP_CT - MFC QP Status Count

MFC_BITSTREAM_BYTECOUNT_SLICE - Bitstream Output Byte Count Per Slice Report Register

MFC_BITSTREAM_SE_BITCOUNT_SLICE - Bitstream Output Bit Count for the last Syntax Element Report Register

MFX_PAK_ERROR Register

MFX_PAK_WARNING Register

MFX_VP8_CNTRL_MASK - Reported BitRateControl parameter Mask

MFX_VP8_CNTRL_STATUS - Reported BitRateControl parameter Status

MFX_VP8_FRM_BYTE_CNT - Reported Final Bitstream Byte Count

MFX_VP8_FRM_ZERO_PAD - Reported Frame Zero Padding Byte Count

MFX_VP8_BRC_DQindex - Reported BitRateControl DeltaQindex

MFX_VP8_BRC_DLoopFilter - Reported BitRateControl DeltaLoopFilter

MFX_VP8_BRC_CumulativeDQindex01 - Reported BitRateControl CumulativeDeltaQindex and Qindex 01

MFX_VP8_BRC_CumulativeDQindex23 - Reported BitRateControl CumulativeDeltaQindex and Qindex 23

MFX_VP8_BRC_CumulativeDLoopFilter01 - Reported BitRateControl CumulativeDeltaLoopFilter and LoopFilter

MFX_VP8_BRC_CumulativeDLoopFilter23 - Reported BitRateControl CumulativeDeltaLoopFilter and LoopFilter

MFX_VP8_BRC_Convergence_Status - Reported BitRateControl Convergence Status

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

121

MMIO Interface

A set of registers are defined and accessible through MMIO interface to serve multiple purposes:

 Use as Status register for Bit Rate Control

 Use for Context Switch in Multipass

Description

Type

Address

Offset

Dec/Enc

MFX_VP8_CNTRL_MASK

BitRateControl parameter Mask

12900

Enc

MFX_VP8_CNTRL_STATUS

BitRateControl parameter Status

12904

Enc

MFX_VP8_FRM_BYTE_CNT

Final Bitstream Byte count

12908

Enc

MFX_VP8_FRM_ZERO_PAD

Final Bitstream Zero Padding Byte

count

1290B

Enc

MFX_VP8_BRC_DQindex

BitRateControl Delta Qindex

12910

Enc

MFX_VP8_BRC_DLoopFilter

BitRateControl Delta LoopFilter

12914

Enc

MFX_VP8_BRC_CumulativeDQindex01

BitRateControl Cumulative Delta

Qindex for Seg0/1

12918

Enc

MFX_VP8_BRC_CumulativeDQindex23

BitRateControl Cumulative Delta

Qindex for Seg2/3

1291C

Enc

MFX_VP8_BRC_CumulativeDLoopFilter01

BitRateControl Cumulative Delta

LoopFilter for Seg0/1

12920

Enc

MFX_VP8_BRC_CumulativeDLoopFilter23

BitRateControl Cumulative Delta

LoopFilter for Seg2/3

12924

Enc

MFX_VP8_BRC_Convergence_Status

BitRateControl Convergence Status

12928

Enc

MFX_VP8_DEBUG_CPBAC0_Bottom

CPBAC0 engine Bottom State

1292C

Enc

MFX_VP8_DEBUG_CPBAC0_RangeCount

CPBAC0 engine Range/Count State

12930

Enc

MFX_VP8_DEBUG_CPBAC0_Misc

CPBAC0 engine Misc. States

12934

Enc

MFX_VP8_DEBUG_CPBAC1_Bottom

CPBAC1 engine Bottom State

12938

Enc

MFX_VP8_DEBUG_CPBAC1_RangeCount

CPBAC1 engine Range/Count State

1293C

Enc

MFX_VP8_DEBUG_CPBAC1_Misc

CPBAC1 engine Misc. States

12940

Enc

Media VDBOX

122

Doc Ref # IHD-OS-KBL-Vol 8-1.17

The following registers are the same as above except they have a different Address Offset. They are used

if the second VDbox (VP8 Encoder) exists.

Description

Type

Address

Offset

Dec/Enc

MFX_VP8_CNTRL_MASK

BitRateControl parameter Mask

1C900

Enc

MFX_VP8_CNTRL_STATUS

BitRateControl parameter Status

1C904

Enc

MFX_VP8_FRM_BYTE_CNT

Final Bitstream Byte count

1C908

Enc

MFX_VP8_FRM_ZERO_PAD

Final Bitstream Zero Padding Byte

count

1C90B

Enc

MFX_VP8_BRC_DQindex

BitRateControl Delta Qindex

1C910

Enc

MFX_VP8_BRC_DLoopFilter

BitRateControl Delta LoopFilter

1C914

Enc

MFX_VP8_BRC_CumulativeDQindex01

BitRateControl Cumulative Delta

Qindex for Seg0/1

1C918

Enc

MFX_VP8_BRC_CumulativeDQindex23

BitRateControl Cumulative Delta

Qindex for Seg2/3

1C91C

Enc

MFX_VP8_BRC_CumulativeDLoopFilter01

BitRateControl Cumulative Delta

LoopFilter for Seg0/1

1C920

Enc

MFX_VP8_BRC_CumulativeDLoopFilter23

BitRateControl Cumulative Delta

LoopFilter for Seg2/3

1C924

Enc

MFX_VP8_BRC_Convergence_Status

BitRateControl Convergence Status

1C928

Enc

MFX_VP8_DEBUG_CPBAC0_Bottom

CPBAC0 engine Bottom State

1C92C

Enc

MFX_VP8_DEBUG_CPBAC0_RangeCount

CPBAC0 engine Range/Count State

1C930

Enc

MFX_VP8_DEBUG_CPBAC0_Misc

CPBAC0 engine Misc. States

1C934

Enc

MFX_VP8_DEBUG_CPBAC1_Bottom

CPBAC1 engine Bottom State

1C938

Enc

MFX_VP8_DEBUG_CPBAC1_RangeCount

CPBAC1 engine Range/Count State

1C93C

Enc

MFX_VP8_DEBUG_CPBAC1_Misc

CPBAC1 engine Misc. States

1C940

Enc

Media VDBOX

Doc Ref # IHD-OS-KBL-Vol 8-1.17

123

Row Store Sizes and Allocations

AVC

VC1

MPEG2

JPEG

ENC

SEC ENC

vin_vmx_pixcoefind_

addr[31:6]

Bitstream

VDS COEF

Orig Pix

BSP data

vin_vmx_mvbsdrs_

addr[31:6]

VAD BSD

VMD RS

VDS MV

MPC MV

vin_vmx_mpcildbmpr_

addr[31:6]

VAM MPR

VDS ILDB

MPC RS

vin_vmx_dmv*_

addr[31:6]

VAM DMV

VCP DMV

vin_vmx_bp_addr

[31:0]

VCP BP

MPEG2 VLD Decoding Mode :

use BSD Row Store only, and

MPEG2 IT Decoding Mode :

MPEG2 IT mode does not need row-store

JPEG VLD Decoding Mode : no row store is needed